Olostep / Scrape page with LLM extraction

Request parameters

• actions body · optional

  Actions to perform on the page before getting the content.

• country body · optional

  Residential country to load the request from. Supported values are: - US (United States) - CA (Canada) - IT (Italy) - IN (India) - GB (England) - JP (Japan) - MX (Mexico) - AU (Australia) - ID (Indonesia) - UA (UAE) - RU (Russia) - RANDOM Some operations, like scraping Google Search and Google News, support all countries.

• formats body · optional

  Formats in which you want the content.

• links_on_page body · optional

  With this option, you can get all the links present on the page you scrape. Links are always returned as absolute URLs.

• llm_extract body · required
• max_age body · optional

  Maximum acceptable age of cached content, in seconds. When a matching scrape already exists and is newer than max_age seconds, Olostep returns the stored result instead of launching a new browser scrape. Defaults to 0 (always scrape fresh). In the dashboard playground, the default is 86400 (24 hours). The maximum allowed value is 604800 (7 days). See the Caching section in the Scrapes feature docs for details.

• metadata body · optional

  User-defined metadata. Not supported yet

• remove_class_names body · optional

  List of class names to remove from the content.

• remove_css_selectors body · optional

  Option to remove certain CSS selectors from the content. Optionally, you can also pass a JSON stringified array of specific selectors you want to remove. The CSS selectors removed when this option is set to default are ['nav','footer','script','style','noscript','svg',[role=alert],[role=banner],[role=dialog],[role=alertdialog],[role=region][aria-label*=skip i],[aria-modal=true]]

• remove_images body · optional

  Option to remove images from the scraped content. Defaults to false.

• screen_size body · optional

  Configuration for screen size. Preset dimensions are available through screen_type: desktop (1920x1080), mobile (414x896), or default (768x1024).

• screenshot body · optional
• transformer body · optional

  Specify the HTML transformer to use, if any. Postlight's Mercury Parser library is used to remove ads and other unwanted content from the scraped content.

• url_to_scrape body · required

  The URL to start scraping from.

• wait_before_scraping body · optional

  Time to wait in milliseconds before starting the scraping.