I am trying to scrape a webpage, however the HTML that playwright is producing in Python does not match what I can inspect in DevTools. I just need a little help with this code.
#import playwright
from playwright.sync_api import sync_playwright
url = '[login to view URL]'
#open url
with sync_playwright() as p:
browser = [login to view URL](headless=False)
page = browser.new_page()
#enable javascript
[login to view URL](url)
#enable javascript
#load the page and wait for the page to load
page.wait_for_load_state("networkidle")
#get the html content
html = page.inner_html("body")
print(html)
#close browser
[login to view URL]()
I'm attempting to find these elements
<div class="block">
<div class="basic">
<p class="shop">Nakano</p>
<p class="itemno">sp4d-00HXGIKJ (0200343351)</p>
</div>
<div class="pic">
<div class="thum">
<a href="/order/detailPage/item?itemCode=1084122883&ref=list&categoryCode=07&keyword=naruto&lang=en">
<img src="[login to view URL] ">
</a>
</div>
</div>
<div class="title">
<p>
<a href="/order/detailPage/item?itemCode=1084122883&ref=list&categoryCode=07&keyword=naruto&lang=en">
Plex DPCF Haruno Sakura Reboru ring of the eyes
</a>
</p>
</div>
<div class="addalert addalert-text-en">
<a href="javascript:addMypageList('1084122883', 1);">Alert</a>
</div>
<div class="soldout">
<p>Sold Out</p>
</div>
<div class="price">
<p>
20,000 yen
</p>
</div>
<div class="price_range">
</div>
<div class="category">
<p>
</p>
</div>
</div>
Essentially the title, price, link, etc. Normally I don't have difficulties with that part, but I cannot figure out how to get playwright to load that so I can parse through the response with BeautifulSoup. If you know how to do this with Selenium or Requests, I will take that as well.
Hello sir, I am an expert in web scraping and automation. I can scrap any website and organize data in excel or csv file.
I can handle this issue and me your code working fine.
Please contact me for more details.
Thank you
$20 USD 1 gün içinde
5,0 (10 değerlendirme)
3,8
3,8
6 freelancers are bidding on average $23 USD for this job
Hi,Hope you are doing well!
I have checked your requirement, yes I can do it, because I already work on similar requirement in past.
If you have interested in my proposal, please contact me.
I will do my best in order to complete your project.
Thanks for your attention.
Best regards
Faizan Maqbool
Hello
I read ur description very carefully and i have experienced this kind of project 2 months ago
I can finish ur project on time you want.
I have gone through your project description.
I find myself as a perfect fit for this job.
I am working as a Python Developer from last 2 year.
Some of my expertise is in the fields:
1. Web Scraping/Web Automation - Selenium, Scrapy, Requests, Beautifulsoup
2. AI and ML
3. Web Designing
4. Wordpress
5. Data Science
6. C/C++
7. SQL
I will be available 24/7 to assist you during the project.
So lets discuss more about it over chat.
Best Regard