Skip to content

Problems with parser api #8

@brian-blivot

Description

@brian-blivot

Background

The parsing process has two stages: loading the html and parsing the html.

The api allows the supplied parser to be either a function or an object (map) containing a mixture of cheerio selectors and functions.

The parser may also contain a fetch function to customise the loading of the html page. The fetch function is called during the loading stage.

Problems

  1. If the parser is a function, it cannot provide a fetch function
  2. If the parser is an object, all keys in the object are processed by the parser during the parsing stage; in particular, any fetch function is called (again) during the parsing stage.

Fixes

  1. Allow the parser to provide an explicit parse function.
  2. When parsing using an object, include only keys that are defined in the api. Other keys, such as fetch and parse, are ignored.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions