Write Python script to import 1.8M rows of CSV data
Bütçe $10-30 USD
I have the following table in a MySQL 2nd Gen 5.7 InnoDB :
CREATE TABLE [login to view URL] (
id INT(11) NOT NULL AUTO_INCREMENT,
name VARCHAR(256) DEFAULT NULL,
source VARCHAR(64) DEFAULT NULL,
price INT(11) NOT NULL,
year INT(11) NOT NULL,
make VARCHAR(64) NOT NULL DEFAULT '',
model VARCHAR(64) NOT NULL DEFAULT '',
mileage INT(11) NOT NULL,
transmission VARCHAR(64) DEFAULT NULL,
num_cylinders INT(11) DEFAULT NULL,
drive_type VARCHAR(64) DEFAULT NULL,
body_type VARCHAR(64) DEFAULT NULL,
fuel_type VARCHAR(64) DEFAULT 'Other',
title_type VARCHAR(64) DEFAULT NULL,
vin VARCHAR(64) DEFAULT NULL,
trim VARCHAR(64) DEFAULT NULL,
color VARCHAR(128) DEFAULT NULL,
location VARCHAR(256) DEFAULT NULL,
source_id VARCHAR(24) DEFAULT '',
url VARCHAR(512) DEFAULT '',
date_listed DATETIME DEFAULT NULL,
date_found DATETIME DEFAULT NULL,
date_updated DATETIME DEFAULT NULL,
num_doors INT(11) DEFAULT NULL,
date_analyzed DATETIME DEFAULT NULL,
seller_type VARCHAR(15) DEFAULT NULL,
details LONGTEXT DEFAULT NULL,
PRIMARY KEY (id)
)
I have a CSV file of 1.8 million rows that I need to have inserted into this database. I need a Python script to do this for me.
The CSV has the following columns:
name,
year,
make,
model,
trim,
mileage,
price,
transmission,
num_cylinders,
drive_type,
fuel_type,
body_type,
color,
location,
url,
sp_url,
date_posted,
date_found,
vin,
title_type,
source
The sp_url column from the CSV should not be imported.
All other columns from the CSV need to be imported.
The database columns num_doors, date_analyzed, seller_type, details, and source_id should all import NULL.
Some of the fields have double quotes " in them. These need to be removed.
Some of the fields are empty / blank. These need to be imported as NULL.
I could not attach the CSV file here because it is too large (700 MB).
Seçilen:
I can provide you a simple Python script that will import your CSV into MySQL database. All I need to start is a sample CSV from you.
Bu iş için 10 freelancer ortalamada $32 teklif veriyor
Nice to meet you. Very interested in your project. I have good talents and skills of python and i can import data from csv. We can share more details of work via chat. I will try to my best for work. Fast speed, Best q Daha Fazla
Hello How are you I have checked your description I am a python expert and that is not a problem for me to edit python script with 1.8M rows data . It is the best proper for me to work on this project , so i am su Daha Fazla
Hi, Nice to meet you. With 6+ years of experience in Web development, I developed a lot of big and small websites using HTML & PHP(Laravel, Codeigniter, Yii) & Python(Django, Flask) & JS(Angular, Node, React, Vue) & Daha Fazla
Hi~ I have read your description carefully and checked your all rquirements fully like following: [login to view URL] sp_url column from the CSV should not be imported. [login to view URL] other columns from the CSV need to be imported. [login to view URL] da Daha Fazla
i have experience in working with database and csv in python of 10mn rows, I can achieve thos task easily.
Hi, I have a good typing speed and can perform your task with ease. I have gone through your details already and can perform your task on priority basis. please share details once.
Hi, I can process the file with a python3 script, with or without sql framework like pewee or sqlalchemy.
Hi.. This is pretty straight forward. Probably the most time consuming thing will be ensuring that the db module (and other pip installs) are easy to install on your system (with what version of python?) prior to run Daha Fazla
Hi, I already made it. Do you need any kind of validation in the CSV fields? My other question is if you will send the CSV and do you expect just the script or the database to be imported? Let me know.
