How to Scrape Web Pages Effectively with Clay

SaaS Tools and Apps
Surfer AI - Best All-in-one Assistant

- Your article will be ready in less than 20 minutes and it will be 10 times cheaper than using a dedicated writer.
- Create ready-to-rank articles in minutes with Surfer AI.
- Research, write, and optimize across industries with the click of a button.

Really Effortless. No prior information essential
Time Needed:
~five minutes

World wide web scraping is a approach that makes it possible for you to extract publicly accessible information on-line.

For numerous companies, it has turn out to be an integral device for gathering details, obtaining insights, and generating smarter choices. For instance, income pros extract information from sites to check a competitor’s pricing, create a prospects record, or carry out marketplace study. 

As a result, getting the proper scraping device can make all the distinction. That is exactly where Clay actions in.

Clay is an AI-powered device developed exclusively to support active pros like you. Aside from scraping information, it also aids you uncover, organize, and deepen your connections across distinct platforms.  

In this guidebook, I will stroll you by way of how to make the most of Clay, from original setup to seamlessly incorporating information to your workspace.

By the finish, you must have your scraped information prepared for enrichment, as proven under:

scrape companies from Crunchbasescrape companies from Crunchbase

Scraping Information from Any Internet site Making use of Clay

one) Set up Clay Extension

Assuming you presently have an lively account on Clay, go to the Chrome World wide web Retailer and seem for the Clay extension. Click ‘Add to Chrome’ to set up.

install Clay extensioninstall Clay extension

two) Open Any Internet site Web page

For this tutorial, we’ll use Crunchbase as an illustration. Login to the internet site and click ‘Companies’ to see a record of a variety of firms.

find companies from Crunchbasefind companies from Crunchbase

three) Initiate Clay for Information Scraping

Click on the Clay icon situated in your toolbar. This action will prompt a settings box to seem. 

scrape page using Clayscrape page using Clay

Right here, you have 3 methods to scrape information. 

The 1st 1 is by way of an autodetected record. This approach maps the information to a table instantly when you click the Clay icon.

Clay autodetected listsClay autodetected lists

The 2nd is by way of Clay’s pre-produced recipe. Right here, information is instantly mapped to a table primarily based on attributes like title, description, picture, and so forth. If a recipe is not accessible for a internet site, you can click ‘Ask Clay to map this web page.’ Clay will notify you when the recipe is prepared.

ask Clay to map pageask Clay to map page

The third 1 is to generate a recipe. This approach lets you map pages and lists as you choose, rather than relying on autodetection.

In this guidebook, we’re going to scrape information by generating our very own Clay recipe. 

three) Include A Listing to Clay Recipe

In the settings pane, click ‘Select Information.’

select data from Crunchbaseselect data from Crunchbase

Then, select ‘Create Recipe and Choose Information.’ Enter the title your recipe.

create Clay recipe and select datacreate Clay recipe and select data

To decide on information, click ‘Select information to include from page’ followed by ‘Select a list’ from the dropdown.

select data to add from Crunchbaseselect data to add from Crunchbase

You will require to decide on two things from the table. This will support Clay realize your table’s construction.

scrape companies from Crunchbasescrape companies from Crunchbase

four) Include Attributes to Your Listing

Click ‘Select a record item attribute to include.’

Hover above the information you want, like a emblem, and decide on it. Identify this attribute – we’ll contact it ‘Logo’ for this illustration. Specify its information variety, in this situation, ‘image,’ and then click ‘Save Alterations.’ You will see a preview of the table with your new attribute.

create attributes on scraped datacreate attributes on scraped data

To include a lot more attributes, click ‘Add attribute’ and repeat the approach. In our illustration, I have extra attributes for the business title, description, and area.

add attribute on scraped dataadd attribute on scraped data

five) Transfer the Scraped Information to Clay

You have two alternatives to include your table to Clay.

1st, you can export the table as a CSV file and then import it into Clay.

export scraped data from Crunchbaseexport scraped data from Crunchbase

2nd, you can right click ‘Add to Workspace’ for Clay to instantly make a table with the scraped information.

add Crunchbase data to Clay's workspaceadd Crunchbase data to Clay's workspace

For simplicity, we’ll select the 2nd alternative in this tutorial.

Soon after clicking the ‘Add to Workspace’ button, you must now see the table on your Clay workspace.

add Crunchbase companies to Clayadd Crunchbase companies to Clay

What’s Up coming? 

Clay makes it possible for any person to scrape information on sites with out coding information. 

But information scraping is just the start off. The subsequent phase is to enrich information and make them a lot more helpful.

Clay has comprehensive integration abilities to uncover deeper insights. For instance, you can leverage the device to discover the e mail addresses of the determination makers, understand the marketplace trends, get the LinkedIn profiles of firms, and so a lot a lot more.

So, I propose you investigate these functions. Right here are a number of tutorials you can comply with:

Lastly, we’re keen to know your ideas, accomplishment stories, or even problems encountered with Clay. Please do share them in the feedback!