{"page":"<link rel=\"stylesheet\" href=\"https://lessonplanet.com/assets/packs/css/resources-572d6a42.css\" />\n<link rel=\"stylesheet\" href=\"https://lessonplanet.com/assets/packs/css/lp_boclips_stylesheets-f4d0de30.css\" media=\"all\" />\n<div data-title='Web Scraping Tutorial with Scrapy and Python for Beginners - Understanding Selectors and Navigating Pages' data-url='/boclips/videos/63eb8a36927e335287bef656' data-video-url='/boclips/videos/63eb8a36927e335287bef656' id='bo_player_modal'>\n<div class='boclips-resource-page modal-dialog panel-container'>\n<div class='react-notifications-root'></div>\n<div class='rp-header'>\n<div class='rp-type'>\n<i aria-hidden='true' class='fai fa-regular fa-circle-play'></i>\nVideo\n</div>\n<h1 class='rp-title' id='video-title'>\nWeb Scraping Tutorial with Scrapy and Python for Beginners - Understanding Selectors and Navigating Pages\n</h1>\n<div class='rp-actions'>\n<div class='mr-1'>\n<a class=\"btn btn-success\" data-posthog-event=\"Signup: LP Signup Activity\" data-posthog-location=\"body_link_boclips\" data-remote=\"true\" href=\"/subscription/new\"><span><span>Get Free Access</span><span class=\"\"> for 10 Days</span><span>!</span></span></a>\n</div>\n</div>\n</div>\n<div class='rp-body'>\n<div class='rp-info'>\n<div aria-label='Hide resource details' class='rp-hide-info' role='button' tabindex='0'>&times;</div>\n<i aria-label='Expand resource details' class='rp-expand-info fai fa-solid fa-up-right-and-down-left-from-center' role='button' tabindex='0'></i>\n<i aria-label='Compress resource details' class='rp-compress-info fai fa-solid fa-down-left-and-up-right-to-center' role='button' tabindex='0'></i>\n<div class='rp-rating'>\n<span class='resource-pool'>\n<span class='pool-label'>Publisher:</span>\n<span class='pool-name'>\n<span class='text'><a data-publisher-id=\"30356011\" href=\"/search?publisher_ids%5B%5D=30356011\">Curated Video</a></span>\n</span>\n</span>\n</div>\n<div class='rp-description'>\n<span class='short-description'>In this video, you will learn how to find and use selectors to scrape data from Amazon.com's search results. The instructor demonstrates how to inspect the URL and use query parameters to search for specific items. The video covers how...</span>\n<span class='full-description hide'>In this video, you will learn how to find and use selectors to scrape data from Amazon.com's search results. The instructor demonstrates how to inspect the URL and use query parameters to search for specific items. The video covers how to select parent and child elements using CSS selectors and how to navigate to the next page using a pagination button. The instructor provides valuable tips for handling variations in product names and versions.</span>\n</div>\n<div class='action-container flex justify-between'>\n<button aria-expanded='false' aria-label='Read more description' class='rp-full-description' type='button'>\n<i class='fai fa-solid fa-align-left'></i>\n<span id='read_more'>Read More</span>\n</button>\n<div class='rp-report'>\n</div>\n</div>\n<div aria-labelledby='resource-details-heading' class='rp-info-section'>\n<h2 class='title' id='resource-details-heading'>Resource Details</h2>\n<div class='rp-resource-details clearfix'>\n<div class='detail'>\n<dl>\n<dt>Curator Rating</dt>\n<dd><span class=\"star-rating\" aria-label=\"4.0 out of 5 stars\" role=\"img\"><i class=\"fa-solid fa-star text-action\" aria-hidden=\"true\"></i><i class=\"fa-solid fa-star text-action\" aria-hidden=\"true\"></i><i class=\"fa-solid fa-star text-action\" aria-hidden=\"true\"></i><i class=\"fa-solid fa-star text-action\" aria-hidden=\"true\"></i><i class=\"fa-regular fa-star text-action\" aria-hidden=\"true\"></i></span></dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt class=\"educator-rating-title\">Educator Rating</dt><dd><div class=\"educator-rating-details\" data-path=\"/educator_ratings/rrp_data?resourceable_id=40994&amp;resourceable_type=Boclips%3A%3AVideoMetadata\"><span class=\"not-yet-rated\">Not yet Rated</span></div></dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt>Media Length</dt>\n<dd>15:42</dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt>Grade</dt><dd title=\"Grade\">Higher Ed</dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt>Subjects</dt><dd><span><a href=\"/search?grade_ids%5B%5D=259&amp;search_tab_id=1&amp;subject_ids%5B%5D=378077\">STEM</a></span></dd><dd class=\"text-muted\"><i class=\"fa-solid fa-lock mr5\"></i>3 more...</dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt>Media Type</dt><dd><span><a href=\"/search?grade_ids%5B%5D=259&amp;search_tab_id=2&amp;type_ids%5B%5D=4543647\">Instructional Videos</a></span></dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt>Source:</dt>\n<div class='preview-source' data-animation='true' data-boundary='.rp-info' data-container='.rp-resource-details' data-html='false' data-title='Instructional videos for technology and computing.' data-trigger='hover focus'>\n<span>Packt -  Programming</span>\n<i aria-hidden='true' class='fa-solid fa-circle-info channel-tooltip-icon' id='channel-tooltip'></i>\n</div>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt>Date</dt>\n<dd>2023</dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<i aria-hidden='true' class='fai fa-solid fa-language'></i>\n<dt>Language</dt><dd>English</dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt>Audiences</dt><dd><span><a href=\"/search?audience_ids%5B%5D=371079&amp;grade_ids%5B%5D=259&amp;search_tab_id=1\">For Teacher Use</a></span></dd><dd class=\"text-muted\"><i class=\"fa-solid fa-lock mr5\"></i>2 more...</dd>\n</dl>\n</div>\n<div class='detail'>\n<dl>\n<dt>Usage Permissions</dt><dd>Fine Print: Educational Use</dd>\n</dl>\n</div>\n</div>\n</div>\n<div aria-labelledby='additional-materials-heading' class='rp-info-section'>\n<h2 class='title' id='additional-materials-heading'>Additional Materials</h2>\n<div class='additional-material'>\n<i aria-hidden='true' class='fai fa-solid fa-lock'></i>\n<a class=\"text-muted\" title=\"Video Transcript\" data-html=\"true\" data-placement=\"bottom\" data-trigger=\"click\" data-content=\"<div class=&quot;text-center py-2&quot;><a class=&quot;bold&quot; href=&quot;/auth/users/sign_in&quot;>Sign in</a> or <a class=&quot;bold text-danger&quot; data-posthog-event=&quot;Signup: LP Signup Activity&quot; data-posthog-location=&quot;body_link_boclips&quot; data-remote=&quot;true&quot; href=&quot;/subscription/new&quot;>Join Now</a></div>\" data-title=\"Get Full Access\" data-container=\"body\" rel=\"popover\" tabindex=\"0\" href=\"/subscription/new\">Video Transcript</a>\n</div>\n<div class='additional-material'>\n<i aria-hidden='true' class='fai fa-solid fa-lock'></i>\n<a class=\"text-muted\" title=\"Video Preview\" data-html=\"true\" data-placement=\"bottom\" data-trigger=\"click\" data-content=\"<div class=&quot;text-center py-2&quot;><a class=&quot;bold&quot; href=&quot;/auth/users/sign_in&quot;>Sign in</a> or <a class=&quot;bold text-danger&quot; data-posthog-event=&quot;Signup: LP Signup Activity&quot; data-posthog-location=&quot;body_link_boclips&quot; data-remote=&quot;true&quot; href=&quot;/subscription/new&quot;>Join Now</a></div>\" data-title=\"Get Full Access\" data-container=\"body\" rel=\"popover\" tabindex=\"0\" href=\"/subscription/new\">Video Preview</a>\n</div>\n</div>\n<div aria-labelledby='additional-tags-heading' class='rp-info-section'>\n<h2 class='title' id='additional-tags-heading'>Additional Tags</h2>\n<div class='clearfix'>\n<div class='details-list keyterms' data-identifier='Boclips::VideoDecorator63eb8a36927e335287bef656' data-type='keyterms'>web scraping, scraping, scrapy, python, website, dynamic website scraping, pycharm 2021.1.3, visual studio code 2022, python 3.10 and 3.8, scrapy 2.6.1, windows 10, ubuntu 20.0.2, mongodb 5.0.2, scrapy-playwright 0.0.12, json, csv, xlsx (excel), xml, search term, base url, hit enter, color addition, scrapy shell, dev item, search results, 1st result, scroll horizontally, response css, pagination item, result inside, parent selector, item, results, 48 results, check, result, scroll, equals, back, response, element, amazon, selector, parent, product, span, enumerate, find</div>\n<div class='keyterms-toggle-buttons' data-identifier='Boclips::VideoDecorator63eb8a36927e335287bef656'>\n<button aria-expanded='false' class='more btn-link' type='button'>\n<span>Show More</span>\n<i aria-hidden='true' class='fa-solid fa-caret-down ml5'></i>\n</button>\n<button aria-expanded='true' class='less btn-link' style='display: none;' type='button'>\n<span>Show Less</span>\n<i aria-hidden='true' class='fa-solid fa-caret-up ml5'></i>\n</button>\n</div>\n</div>\n</div>\n<div aria-labelledby='classroom-considerations-heading' class='rp-info-section'>\n<h2 class='title' id='classroom-considerations-heading'>Classroom Considerations</h2>\n<div class='classroom-considerations'><div class='fai fa-solid fa-bell'></div>Best For: Explaining a topic</div><div class='classroom-considerations'><div class='fai fa-solid fa-bell'></div>Video is ad-free</div> \n</div>\n<div aria-labelledby='educator-ratings-heading' class='rp-info-section'>\n<h2 class='title sr-only' id='educator-ratings-heading'>Educator Ratings</h2>\n<div id=\"educator-ratings-root\"></div><div id=\"all-educator-ratings-root\"></div><div id=\"educator-rating-form-root\"></div>\n</div>\n</div>\n<div class='rp-resource'>\n<div aria-label='Show resource details' class='rp-show-info' role='button' tabindex='0'>\n<i class='fai fa-solid fa-align-left'></i>\nShow resource details\n</div>\n<div aria-label='Video player' class='player' id='player-wrapper' role='region'>\n<div class='relative container mx-auto' id='lp-boclips-visitor-thumbnail'>\n<a class=\"block\" data-html=\"true\" data-placement=\"bottom\" data-trigger=\"click\" data-content=\"<div class=&quot;text-center py-2&quot;><a class=&quot;bold&quot; href=&quot;/auth/users/sign_in&quot;>Sign in</a> or <a class=&quot;bold text-danger&quot; data-posthog-event=&quot;Signup: LP Signup Activity&quot; data-posthog-location=&quot;body_link_boclips&quot; data-remote=&quot;true&quot; href=&quot;/subscription/new&quot;>Join Now</a></div>\" data-title=\"Get Full Access\" data-container=\"body\" rel=\"popover\" tabindex=\"0\" aria-label=\"Play video: Web Scraping Tutorial with Scrapy and Python for Beginners - Understanding Selectors and Navigating Pages\" href=\"/subscription/new\"><img class=\"resource-img img-thumbnail img-responsive z-10 lp-boclips-thumbnail w-full h-full lozad\" alt=\"Web Scraping Tutorial with Scrapy and Python for Beginners - Understanding Selectors and Navigating Pages\" title=\"Web Scraping Tutorial with Scrapy and Python for Beginners - Understanding Selectors and Navigating Pages\" onError=\"handleImageNotLoadedError(this)\" data-default-image=\"https://statictemp.lp.lexp.cloud/images/attachment_defaults/resource/large/missing.png\" data-src=\"https://cdnapisec.kaltura.com/p/1776261/thumbnail/entry_id/1_i87c68gf/width/250/vid_slices/3/vid_slice/1\" width=\"315\" height=\"220\" src=\"data:image/png;base64,R0lGODlhAQABAAD/ACwAAAAAAQABAAACADs\" />\n<span aria-hidden='true' class='flex justify-center items-center bg-white rounded-full w-16 h-16 absolute top-1/2 left-1/2 -mt-8 -ml-8 cursor-pointer z-0 border-2 border-primary drop-shadow-md lp-boclips-thumbnail-playBtn'>\n<i class='fa-solid fa-play text-primary text-3xl ml-1 drop-shadow-xl'></i>\n</span>\n</a></div>\n</div>\n</div>\n</div>\n</div>\n</div>\n"}