Package org.jweaver.crawler.internal.write

Interface JWeaverWriter

All Known Implementing Classes:
JWeaverFileWriter
public interface JWeaverWriter
The JWeaverWriter interface defines methods for processing and writing the results of the web crawling process. Implementations of this interface are responsible for handling successful crawled pages, error information, and connection maps generated during the crawling process.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    processConnectionMap(String baseUri, List<Connection> connections, ExportConfig exportConfiguration)
    Processes connection map information generated during crawling and writes it using the provided export configuration.
    void
    processErrors(String baseUri, List<NodeError> nodeErrorList, ExportConfig exportConfiguration)
    Processes errors encountered during crawling and writes error information using the provided export configuration.
    void
    processSuccess(SuccessResultPage successResultPage, ExportConfig exportConfiguration)
    Processes a successfully crawled page and writes the result using the provided export configuration.

  • Method Details

    • processSuccess

      void processSuccess(SuccessResultPage successResultPage, ExportConfig exportConfiguration)
      Processes a successfully crawled page and writes the result using the provided export configuration.
      Parameters:
      successResultPage - The success result page containing information about the crawled page.
      exportConfiguration - The export configuration specifying how the result should be written.

    • processErrors

      void processErrors(String baseUri, List<NodeError> nodeErrorList, ExportConfig exportConfiguration)
      Processes errors encountered during crawling and writes error information using the provided export configuration.
      Parameters:
      baseUri - The base URI of the page where the errors occurred.
      nodeErrorList - A list of NodeError objects containing information about the errors of failed crawled uris
      exportConfiguration - The export configuration

    • processConnectionMap

      void processConnectionMap(String baseUri, List<Connection> connections, ExportConfig exportConfiguration)
      Processes connection map information generated during crawling and writes it using the provided export configuration.
      Parameters:
      baseUri - The base URI of the page.
      connections - A list of Connection objects representing connections between pages.
      exportConfiguration - The export configuration