-
-
04 Jul 2009 01:44:17 UTC
- Development release
- Distribution: KinoSearch
- Source (raw)
- Browse (raw)
- Changes
- How to Contribute
- Issues (5)
- Testers (13 / 3 / 1)
- Kwalitee
Bus factor: 0- License: perl_5
- Activity
24 month- Tools
- Download (611.03KB)
- MetaCPAN Explorer
- Permissions
- Subscribe to distribution
- Permalinks
- This version
- Latest version
and 1 contributors- Marvin Humphrey <marvin at rectangular dot com>
- Dependencies
- JSON::XS
- Lingua::Stem::Snowball
- Lingua::StopWords
- and possibly others
- Reverse dependencies
- CPAN Testers List
- Dependency graph
NAME
KinoSearch::Docs::Tutorial::BeyondSimple - A more flexible app structure.
DESCRIPTION
Goal
In this tutorial chapter, we'll refactor the apps we built in KinoSearch::Docs::Tutorial::Simple so that they look exactly the same from the end user's point of view, but offer the developer greater possibilites for expansion.
To achieve this, we'll ditch KSx::Simple and replace it with the classes that it uses internally:
KinoSearch::Schema - Plan out your index.
KinoSearch::FieldType::FullTextType - Field type for full text search.
KinoSearch::Analysis::PolyAnalyzer - A one-size-fits-all parser/tokenizer.
KinoSearch::Indexer - Manipulate index content.
KinoSearch::Searcher - Search an index.
KinoSearch::Search::Hits - Iterate over hits returned by a Searcher.
Adaptations to indexer.pl
After we load our modules...
use KinoSearch::Schema; use KinoSearch::FieldType::FullTextType; use KinoSearch::Analysis::PolyAnalyzer; use KinoSearch::Indexer;
... the first item we're going need is a Schema.
A Schema is analogous to an SQL table definition. It instructs other entities on how they should interpret the raw data in an inverted index and interact with it.
The primary job of a Schema is to specify what fields are available and how they're defined. We'll start off with three fields: title, content and url.
# Create Schema. my $schema = KinoSearch::Schema->new; my $polyanalyzer = KinoSearch::Analysis::PolyAnalyzer->new( language => 'en', ); my $type = KinoSearch::FieldType::FullTextType->new( analyzer => $polyanalyzer, ); $schema->spec_field( name => 'title', type => $type ); $schema->spec_field( name => 'content', type => $type ); $schema->spec_field( name => 'url', type => $type );
All of the fields are spec'd out using the "FullTextType" FieldType, indicating that they will be searchable as "full text" -- which means that they can be searched for individual words. The "analyzer", which is unique to FullTextType fields, is what breaks up the text into searchable tokens.
Next, we'll swap our KSx::Simple object out for a KinoSearch::Indexer. The substitution will be straightforward because Simple has merely been serving as a thin wrapper around an inner Indexer, and we'll just be peeling away the wrapper.
First, replace the constructor:
# Create Indexer. my $indexer = KinoSearch::Indexer->new( index => $path_to_index, schema => $schema, create => 1, truncate => 1, );
Next, have the
$indexer
objectadd_doc
where we were having the$simple
objectadd_doc
before:foreach my $filename (@filenames) { my $doc = slurp_and_parse_file($filename); $indexer->add_doc($doc); }
There's only one extra step required: at the end of the app, you must call commit() explicitly to close the indexing session and commit your changes. (KSx::Simple hides this detail, calling commit() implicitly when it needs to).
$indexer->commit;
Adaptations to search.cgi
In our search app as in our indexing app, KSx::Simple has served as a thin wrapper -- this time around KinoSearch::Searcher and KinoSearch::Search::Hits. Swapping out Simple for these two classes is also straightforward:
use KinoSearch::Searcher; my $searcher = KinoSearch::Searcher->new( index => $path_to_index ); my $hits = $searcher->hits( # returns a Hits object, not a hit count query => $q, offset => $offset, num_wanted => $hits_per_page, ); my $hit_count = $hits->total_hits; # get the hit count here ... while ( my $hit = $hits->next ) { ... }
Hooray!
Congratulations! Your apps do the same thing as before... but now they'll be easier to customize.
In our next chapter, KinoSearch::Docs::Tutorial::FieldType, we'll explore how to assign different behaviors to different fields.
COPYRIGHT
Copyright 2005-2009 Marvin Humphrey
LICENSE, DISCLAIMER, BUGS, etc.
See KinoSearch version 0.30.
Module Install Instructions
To install KSx::Simple, copy and paste the appropriate command in to your terminal.
cpanm KSx::Simple
perl -MCPAN -e shell install KSx::Simple
For more information on module installation, please visit the detailed CPAN module installation guide.