From Crawl To Clarity Automating SEO Audits With Node.JS

Transcript

2. Technical Lead, Fly High Media Interested in Speeding up SEO auditing & workflows
3. WANT TO FOLLOW ALONG WITH THE SLIDES?
4. TECHNICAL AUDITS WERE TAKING TOO LONG!
6. TOOLS Web Server LLM of Choice Domain Name
7. HAS ANYONE USED NODE.JS BEFORE?
8. WHAT IS NODE.JS?
9. FREE JAVASCRIPT FRAMEWORK
10. ALLOWS DEVELOPERS TO CREATE CUSTOM TOOLS
11. WHY AUTOMATE SEO AUDITS THIS WAY?
12. REDUCE WORKLOAD
13. CUSTOMISABLE
14. LOW COST
15. WHAT CAN WE AUDIT?
16. HTML STATUS CODES (404, 301, ETC..)
17. CANONICAL TAG ISSUES
18. PAGINATION ISSUES
19. METADATA ISSUES AT SCALE
20. HREFLANG ISSUES
21. XML SITEMAP CHECKS
22. REDIRECT CHAINS & LOOPS
23. IMAGE ALT TEXT
24. URL STRUCTURE
25. PAGE SPEED ISSUES
26. BROKEN INTERNAL/EXTERNAL LINKS
27. SCHEMA MARKUP
28. PAGE SIZE IS LESS THAN 2MB GOOGLE NEW CRAWLING LIMIT - FEB 2026
29. HOW CAN WE BUILD AN AUDITING TOOL?
30. LLMS CAN CREATE THE CODE FOR ME!
32. “I WANT TO CREATE A WEB APP THAT ALLOWS ME TO CRAWL WEBSITES FOR ANY TECHNICAL SEO ISSUES USING NODEJS”
34. USE A CODE EDITOR LIKE VISUAL STUDIO CODE TO CREATE THE DIRECTORY
37. WE NOW NEED TO SETUP A SERVER TO HOST THE APP ON
39. HETZNER SERVER SETUP Use a Server Local to You (Helsinki for UK) Ubuntu Server Shared vCPU or Dedicated vCPU
40. WHAT IS SSH? SSH is a secure digital tunnel that allows you to safely log into a website's server to manage backend files directly.
41. HOW TO ACCESS SSH? Use the Hetzner’s Server Console SSH software e.g. Termius
42. INSTALL APP FROM GITHUB cd ~/apps/ git clone https://github.com/yourusername/seo-crawler.git cd seo-crawler
43. INSTALL NODE DEPENDENCIES npm install
44. RUN THE APP npm run dev
45. WANT TO PUT IT ON A DOMAIN/SUBDOMAIN?
46. REGISTER YOUR DOMAIN
47. POINT THE DOMAIN/SUBDOMAIN TO THE SERVER IP ADDRESS
49. DOMAIN WILL NOW LOAD YOUR SERVER
50. AN EASIER WAY TO SETUP (IF YOU HAVE THIS FEATURE IN CPANEL)
51. BUILT INTO CPANEL
52. INSTALL USING WHM OR ASK YOUR HOSTING COMPANY
54. UPLOAD YOUR NODE.JS APP TO A DIRECTORY ON YOUR SERVER AND MAKE A NOTE OF THE DIRECTORY YOU SAVED IT IN
57. ENTER THE DIRECTORY YOU PUT THE CODE IN
59. SELECT THE DOMAIN / SUBDIRECTORY YOU WANT TO LOAD THE APP FROM
61. ENTER THE STARTUP FILE NORMALLY SOMETHING LIKE SERVER.JS / YOU CAN ASK THE LLM YOU USE TO CREATE THE APP FOR THIS
63. IF USING A DIFFERENT DIRECTORY TO THE ONE YOU’RE LOADING THE APP CODE FROM YOU NEED TO ADD BASE_URL TO THE ENVIRONMENT VARIABLES
65. THEN JUST CLICK ‘CREATE’
69. YOU DON’T HAVE TO USE CHATGPT/ANY OTHER AI
70. Google Antigravity App
71. Windsurf
72. Vercel App
73. IF THERE ARE NO ISSUES YOUR APP SHOULD LOAD UP
74. TROUBLESHOOTING
75. USE AI TOOLS
76. NPM AUDIT WILL FLAG VULNERABILITIES
77. COMMON ISSUES
78. 500 ERROR ON SERVER
79. USE AI TOOLS
82. ASSESS THE TOP 5 PRIORITIES LONG TERM FIXES, QUICK WINS
83. ACCESS TO THE SLIDES SPEAKERDECK.COM/JONATHONROBERTSSEO
84. linkedin.com/in/jonathon-roberts-seo/ flyhighmedia.co.uk jonathon@flyhighmedia.co.uk