If youвЂ™ve not used it before, fire up Chrome and right-click somewhere on a web page and select вЂњInspectвЂќ. Gone are the days of View Source just dumping an entire pageвЂ™s HTML into a plain text editor, we now have powerful tools for navigation, inspection, editing and profiling a page all within the browser. Most modern browsers contain a built in set of tools to aid web developers inspect the source of web sites and debug their code. This post isnвЂ™t intended to be a how-to on programming in Python (as I am a mere novice in such regards) or an all-encompassing guide to web scraping but more as a couple of pointers that might help you along the way. One set of tools IвЂ™ve recently enjoyed using on a recent scraping mission is Google Chrome to investigate a pageвЂ™s markup and the Python programming language along with the Requests, and BeatifulSoup 4 libraries. IвЂ™ve been doing odd bits of web scraping in various job roles using various technologies (VBA macros in an Excel sheet being a particular low point) for about 15 years and over that time there have been some changes that make things a bit more difficult but the tools available have become orders of magnitudes better.

Beautifulsoup get plain text of markup