What to look for when using screaming Frog
1. Crawl the site
2. Save the crawl – save in .seospider file format
3. Check page levels – pages too deep won’t be crawled. In screaming frog levels 4 upward give a record of pages that are too deep. The results will show page levels and page url. You can then decide how to go about linking to these deep pages through new blog articles or content links. Be sure the reference links are valid and relevant.
4. Check crawl errors – googleweb master will give you crawl errors but are often out of date or only part results. A fresh crawl with screaming frog may give you a different result list of 400 errors.
Getting crawl errors with Screaming frog:
- Click on “Response Codes”
- Select “Client Error 4xx” from the filter drop down menu
- Export as CSV
5. Find and fix long descriptions
Meta tags and meta descriptions have a fixed length. Screaming frog allows you to check the lengths for any problems, it also allow searching of data.
Producing a new title with the correct length
- Click on over to “Page Titles” in the top menu.
- Select “Over 70 Characters” from the menu.
- Export as CSV
- Create a new column for your new title a column for new length.
- Add this simple formula to the “new length” column; =LEN(E3)
- Select the formula cell
- Hover over the bottom right corner of the formula cell and wait for the cursor to change.
- Drag the square down the entire column
Do this for “descriptions” too.
- Click on descriptions menu in screaming frog
- Select over 156 characters
- Make a new description column and a new length column
- Add =LEN(E2) to new length column to count the length of your description
6. Look at indexation settings
- to check your indexation settings click on “meta and canonical” menu
- You are looking for:
- Missing canonical tags
- Incorrect Canonical tags (pointing to a different page etc)
- Pages that should be indexed but have a “noindex” tag on them.
- Pages that should NOT be indexed but have no meta tag or have “index”.
7. Find all pages with HTML
word press plug in http://wordpress.org/extend/plugins/cross-linker/ which can help you always internally link and cross link your posts.