Dynamic serialization allows you to speed up your extractions by a significant amount, without the need of manual serialization. Dynamic serialization also makes sure you get all data all the time, regardless if a website happens to introduce hundreds of new items on a bunch.

How it works

  • When you are extracting data on a website, after initial data selection the extension asks you if there are items on other pages. Examples of such pages can be found in the dynamic help menu at the top right of the Sniffie top bar.
  • If there are items on other pages, answer the question as “Yes” and the extension asks you to click on a “next page” link. This libnkcan also be a “load more” button. If you made a mistake, you can cancel by clicking the “Cancel” button.
  • When you click "next page" link and if you have activated advanced mode, you can edit the XPath of the clicked element manually. This is not a requirement, but is sometimes useful, since XPath is one of the factors Sniffie considers when navigating to next page with items. Click "continue" to move on.
  • Now Sniffie starts its table processing normally. When you save the data, all information regarding the next page link/button is also saved to Sniffie.

What happens behind the scenes

When you save data extraction with a dynamic serialization, the information on how to access the “next page” link/button is saved with your extraction. Sniffie then knows how to process multiple pages in a certain category. When Sniffie clicks the link/button you have configured, Sniffie looks at what happens in the page. If new items are loaded, Sniffie keeps on clicking the link. When no new items appear, Sniffie will stop the extraction.

After each click, the new items are saved to your databases and are immediately accessible to you.

If the page changes its layout in a very major way, Sniffie will try to find the next page link/button and reconfigure the extraction automatically. This may not sometimes work,

Troubleshooting

When you create a dynamic sniff, the first time Sniffie doesn’t know if the sniff is really serializable or not. The first page results will appear immediately under "data sources". It will take some time for the rest of the results to appear depending on the amount of pages. In addition, a maximum of 100 pages are dynamically serializable. If you come back to your sniff after a while and the dynamic serialization has failed, please contact Sniffie support! Our data experts will help you to get the data you need!

Did this answer your question?