I need help to transform my large datafiles (ie. between 500,000 and 2,000,000 lines) in csv format with 4 numeric type variables (eg. A, B, C and D) recorded in millisecond time points (format [url removed, login to view] hh:mm:ss.000), structured as follows:
Datetime_Milliseconds A B C D
13.12.2017 00:00:00.176 1 2 3 2
I need a developer to provide me (1) macros, script or some other code or method that will transform the file as described below, and (2) with some guidance about how to edit the specific calculations described below, so that I can modify them myself in future.
1. Create new variables (eg. E) that are derived from calculations involving variables in the datafile.
For example, if A>0, E = C , else E = 0
Datetime_Milliseconds A B C D E
13.12.2017 00:01:176 1 2 3 2 3
2. Create a new datafile that summarises values (as either the maximum, minimum, sum, mean, median or mode) in longer time periods (eg. 1, 5, 30 minutes etc or 1, 2, 8 hours etc.
For example, the mean average of values that occur in a 1 minute period
Datetime_1Minute A B C D E
13.12.2017 00:01 1 3 8 3 6
3. I then need to import variables from other csv files that contain data recorded in the same time period.
For example, import variables F and G from another csv format datafile recorded in the same time period (eg. 1 minute data points)
Datetime_1Minute A B C D E F G
13.12.2017 00:01 1 3 8 3 8 4 8
4. Create new variables (eg. N1) that are derived from calculations using variables in the datafile:
N1 = (A_maximum – A_minimum)/E
Datetime_1Minute A B C D E F G N1
13.12.2017 00:01 1 3 8 3 8 4 8 0.25
The transformed datafiles must be in, or able to be saved or exported into csv or tab-delimited format.
It would be preferable for the solution to be executed using MS Excel and/or MS Access, but this is not strictly essential. The software used does however need to be compatible with Windows 10 and be able to handle very large datasets.
I have some experience in coding but it was a long time ago - so I may need the developer to explain how to modify parts of their solution so that I can make some specific changes (eg. modify the calculation required for creating a new variable (see step #4), choose a different time interval (see step #1) to summarise data or to use median values rather than using the mean etc). Please do not apply for this job if you are unwilling to do this.
Payment will be made after (1) explanation is provided about how to modify certain aspects of the solution as described above, and (2) when the solution can be applied to successfully transform another similar datafile. If this test is unsuccessful, the developer will be given an opportunity to produce another solution. Persons who want to be paid for incomplete work should not apply for this job.
Applicants should indicate the general approach they plan to take (including software) and the approximate amount of time required to produce a solution that I can test.
The selected applicant will be provided with the following to develop a solution upon:
- a csv datafile containing approximately 600,000 lines of data
- another smaller file to import data as described in step #3
- additional calculations to include with steps #2 and #4
Hi, there. My name's Paris and I'm a very experienced Excel developer. All you mention in your brief can be tackled by using a macro-enabled Excel spreadsheet. I will write all necessary code to import and export csv Daha Fazla
Bu iş için 15 freelancer ortalamada €51 teklif veriyor
Hi there..... Warm Greetings We came along with your request for Develop database and we reviewed your project description. We'd like to help you with confidence and satisfying results... We have professionals Daha Fazla
Dear Sir, I'm expert on access and ready to build you an perfect solution well documented [login to view URL] about it will be a good choice. Kind Regards
This project can be very easily done using PHP scripting language. It's possibly to run PHP scripts without having a web server installed simply by typing: [login to view URL] [login to view URL] and the code insi Daha Fazla
IT expert here, with both an academic and professional background that can be verified through my profile. MS Access seems most appropriate for your project, given the data size and operations required. I expect to Daha Fazla
I think I can help you on your project. I believe that a VBA script could solve all steps that you had described. I think that I could do the entire assignment within 4-5 days. My proposal is to do each step in 1 day a Daha Fazla
Database & Data Entry Expert! I can offer you the best service within 1 day. It would be unspeakably excellent ever. Thank you!
I can deliver this project using SQL Server. If you have SQL Server installed in your local, you can use that solution. I will load those 2 data files into SQL Database table and then i will process required through St Daha Fazla