Changes between Version 8 and Version 9 of DynamicDatasources

Timestamp:

04/12/06 16:22:34 (18 years ago)

Author:

grimnes

Comment:

--

DynamicDatasources

@@ -38 +38 @@
  * Create a new class that extends CrawlerBase, and add a constructor that takes a DataSource, and calls super(); and setDataSource with the argument. Use this constructor to return a new crawler instance in your crawlerFactory class.
  * Now the crawlerFactory, DataSource and DataSource factory classes are all finished - only the meat remains, implementing the crawlObjects method.
+ * First you have to decide what configuration options your datasource will take. Have a look at [http://aperture.sourceforge.net/ontology/source.rdfs the aperture datasource schema] for a selection. In my case I will use source:rootURI to specify what RSS feed to crawl. Use the aperture utility method to get this: 
+{{{
+RDFContainer config=source.getConfiguration();
+String root=ConfigurationUtil.getRootUrl(config);
+}}}
+ You are of course free to make up any config properties you want, but then the ConfigurationUtil class might not help you. 
+ * Implementing the crawlObjects method is clearly quite datasource dependent, in my case I add the rome jar and jdom, and copy some example for how to read a feed. Some other hints:
+ ** Gnowsis uses java.util.logging for logging, so to get useful debugging message add this the top of your file: 
+{{{ 
+Logger log=Logger.getLogger(RSSCrawler.class.getName());
+}}}
+ ** The return value of crawlObjects is taken from ExitCode, it has predefined values for you.