Multivariate Regression

Contest Brief

Develop a predictive model from a data set to predict an Score based on several sub-scores (Pr, Pe, Ca, Co, and No). You may eliminate factors if they do not contribute to the model performance.

Do NOT include rank as a factor. Extra credit for developing a predictor for the rank also.

For example a linear regression yields the following: Score = -8.5825 + 0.00112183767 * Pr + 7.868 * Pe + 2.6883 * Ca + 7.9149601767352 * No.

RSquare 0.790245
RSquare Adj 0.783937
Root Mean Square Error 4.557223

To be considered the model must outperform a linear regression in terms of residuals and in terms of performance against a validation data set (which I hold).

Can you develop a better model? Be creative! Teach me something new. Do NOT submit a linear regression.

Please deliver a brief description of the model, technique used, results, and the mathematics.

**ADDED SMALL DATA SET TO ALLOW FURTHER MODEL VALIDATION**

Recommended Skills

Winner
Featured Contest
Post a Contest Like This

Public Clarification Board

  • carowjp
    Contest Holder
    • 11 years ago

    Ranking:
    ---------------
    twistedhardware
    tsendee
    heatherturner
    danielharmon
    hobekyo
    DevDataAnalyst
    priyal1997
    CH1979
    akhilrana690041
    pradan015
    jaquescohen
    dobreiita
    doot1989
    abhishek
    statAnalysis

    • 11 years ago
    1. twistedhardware
      twistedhardware
      • 11 years ago

      Thank you Carowjp .. The file is uploded.

      • 11 years ago
    2. akhilrana690041
      akhilrana690041
      • 11 years ago

      thanks

      • 11 years ago
  • heatherturner
    heatherturner
    • 11 years ago

    Thanks for the feedback on our submissions. From the comments on the winning submission, it looks like the validation data may have been used in training the model. If this is the case the RMSE will be biased (lower than the true error rate). I thought that the intention of providing the validation data was to obtain an independent assessment of the model accuracy, i.e. an idea of how our model would perform in practice.

    • 11 years ago
    View 3 more messages Processing...
    1. Data Analysis Solutions
      Data Analysis Solutions
      • 11 years ago

      First carowjp thanks for the interesting challenge and congratulations to twistedhardware.

      • 11 years ago
    2. Data Analysis Solutions
      Data Analysis Solutions
      • 11 years ago

      Since the rankings have been made public, I take the chance to write a short rebuttal.

      (1) I assume that twistedhardware has not been using test data to build his model.
      (2) In the future it may be reasonable to withhold the target values (in this case the score) in the test set. Then it would be a level playing field and cheating due to overfitting (use of test set in model) or fine tuning of the hyperparameters to find the best fit for the four values is impossible.
      (3) Given the large underperformance of my model I want to point out that it would have been possible to find a set of hyperparameters(grid search) with a nearly perfect fit for the fours test values without using the test data in the model! I am afraid that this kind of optimizing is not useful long term.

      Please let me know if you have any questions. best regards.

      • 11 years ago
  • carowjp
    Contest Holder
    • 11 years ago

    Thank you all for your submissions. The range of RMSE using predictions rounded to integers was 0.87 to 4.61. The top three performances were 0.87 (SVM), 1.12 (Decision Tree), and 1.50 (SVM).

    • 11 years ago
  • dobreiiita
    dobreiiita
    • 11 years ago

    Caroqjp, Would you like to discuss my solutions as well !?

    • 11 years ago
  • jaquescohen
    jaquescohen
    • 11 years ago

    Carowjp, I completely understand it. First, about your regression (linear using Pr, Pe, Ca, No); If used the Least Squares method, then you don’t get the same coefficients and the R-Square is 69.14%. Second, if R-Square is calculated step by step with your data and your regression presented, then it will be at 65.03%; not 79.03% as you say in your description. I have done an Excel with that step by step calculation to prove you that. Second, I have 2 main equations; with the validating data set; they predicted the following scores: (76.83; 68.40; 74.12; 68.40) and (76.28; 68.41; 74.53; 68.41). And resulted in the following errors/resids (-1.83; 4.60; -1.12; 1.60) and (-1.28; 4.59; -1.53; 1.59). I also have a Rank prediction if need although there is a point on the validation data that makes it difficult to achieve reasonable Resids results for Rank as dependent variable.

    • 11 years ago
  • carowjp
    Contest Holder
    • 11 years ago

    Jacques, unfortunately I will be charged a significant fee to extend the contest. I would rather use those funds for a follow-up project. If you simply paste the results of your prediction for the Feb15 dataset here I can see if your approach is competitive to that of your peers.

    • 11 years ago
  • jaquescohen
    jaquescohen
    • 11 years ago

    carowjp, I try to send you the files in all the possible ways I thought but the freelancer.com does not let me. Of corse I would like to participate on the contest; but even if not possible, as I have done a work here I would like you at least to be abble to receive the file and see if it helps you in any way. Can you consider extenting for 1 day just to make possible for the file to be send?

    • 11 years ago
  • carowjp
    Contest Holder
    • 11 years ago

    Hello All, I am still waiting for a clarification from one or two contestants. I will post feedback and award the contest soon.

    • 11 years ago
    1. DevDataAnalyst
      DevDataAnalyst
      • 11 years ago

      ok sir

      • 11 years ago
  • DevDataAnalyst
    DevDataAnalyst
    • 11 years ago

    Hi , please Provide ratings to the work of all contestants , it will be like a feedback of our work

    • 11 years ago
  • dobreiiita
    dobreiiita
    • 11 years ago

    Hello Organizer, can you share the RMSE ranking so that we can have a look at a leader's board !!

    • 11 years ago
  • carowjp
    Contest Holder
    • 11 years ago

    If you have not done so please post the results of your models for predicting the added validation data (4 rows) and their residuals. You can deliver your completed work by dropbox or Google Drive, etc. Thank you for your efforts.

    • 11 years ago
    View 2 more messages Processing...
    1. akhilrana690041
      akhilrana690041
      • 11 years ago

      do we need to send spreadsheet

      • 11 years ago
    2. DevDataAnalyst
      DevDataAnalyst
      • 11 years ago

      i have posted a screenshot of my result , please check

      • 11 years ago
  • akhilrana690041
    akhilrana690041
    • 11 years ago

    Thanks for the work i really like the work hope to see some research from you in future. Working something like this is awesome

    • 11 years ago
  • tsendee
    tsendee
    • 11 years ago

    Check out my submission Entry #28 .

    Thank you!

    • 11 years ago
  • DevDataAnalyst
    DevDataAnalyst
    • 11 years ago

    I got a set of predicted values but cant send it , can you message from your side ?

    • 11 years ago
  • dobreiiita
    dobreiiita
    • 11 years ago

    We have submitted our work in the form of graphs and snippet of predictions.

    • 11 years ago
    View 1 more message Processing...
    1. dobreiiita
      dobreiiita
      • 11 years ago

      Our is a black box model. Hence rather not possible to provide you with an equation. However we can share the code, if you would like to review them before deciding your winner !!

      • 11 years ago
    2. dobreiiita
      dobreiiita
      • 11 years ago

      Added another entry as for the test data, entry #23

      • 11 years ago
  • freelanmohan7
    freelanmohan7
    • 11 years ago

    Hi, How much is the accuracy you are expecting? What are the acceptable error values for you? Would it be ok to use neural network kind of models?

    • 11 years ago
    1. carowjp
      Contest Holder
      • 11 years ago

      Yes you can apply NN.

      • 11 years ago
  • carowjp
    Contest Holder
    • 11 years ago

    **I ADDED SMALL DATA SET (additional CSV file) TO ALLOW FURTHER MODEL VALIDATION**

    • 11 years ago
  • abhishek16041983
    abhishek16041983
    • 11 years ago

    I can't able to upload an MS-Excel 2010 file (only image files are allowed e.g. JPG, PNG, GIF). What should I do?

    • 11 years ago
    View 2 more messages Processing...
    1. carowjp
      Contest Holder
      • 11 years ago

      You could use Google Drive or Dropbox etc

      • 11 years ago
    2. tsendee
      tsendee
      • 11 years ago

      Cool, submitted my experiment with Dropbox url. Check it out!

      • 11 years ago
  • kmittal
    kmittal
    • 11 years ago

    I can deliver as a 'project' for a much lower price.

    • 11 years ago
    1. carowjp
      Contest Holder
      • 11 years ago

      Thanks for that. I am trying as a contest in hopes to generate many ideas,

      • 11 years ago
  • deepakbansal91
    deepakbansal91
    • 11 years ago

    hello, I would like to do this project..tell me submitting procedure !!!!

    • 11 years ago
    1. carowjp
      Contest Holder
      • 11 years ago

      Others are submitting spreadsheets for example.

      • 11 years ago
    2. carowjp
      Contest Holder
      • 11 years ago

      You could use Google Drive or Dropbox etc

      • 11 years ago
  • carowjp
    Contest Holder
    • 11 years ago

    You could use Google Drive or Dropbox etc

    • 11 years ago
  • khanable
    khanable
    • 11 years ago

    I assume that when you say no linear regression models you mean no forms of any general linear model?

    • 11 years ago
    1. carowjp
      Contest Holder
      • 11 years ago

      I am looking for more sophisticated solutions than a simple least squares regression.

      • 11 years ago
  • hobekyo
    hobekyo
    • 11 years ago

    I guess there might be something wrong with your linear regression result. I conducted a multivariate linear regression and the result is different.
    Residuals:
    Min 1Q Median 3Q Max
    -15.1672 -2.8807 0.7144 2.8338 8.0371

    Coefficients:
    Estimate Std. Error t value Pr(>|t|)
    (Intercept) 1.120e+00 5.867e+00 0.191 0.8490
    Pr -4.753e-05 1.644e-03 -0.029 0.9770
    Pe 5.134e+00 7.820e-01 6.565 1.31e-09 ***
    Ca 3.046e+00 1.277e+00 2.385 0.0186 *
    Co 1.656e+00 7.928e-01 2.089 0.0388 *
    No 6.934e+00 5.916e-01 11.722 < 2e-16 ***.
    It's obvious Co is more likely to be 5 in high score observations.

    • 11 years ago
  • twistedhardware
    twistedhardware
    • 11 years ago

    Please check my results on #7 #9 and #10 showing the two predictive models.

    • 11 years ago
  • Danielharman
    Danielharman
    • 11 years ago

    HI, I have earned a master's degree in statistics and have required experience. I would like you to share more details. Thanks

    • 11 years ago
    View 6 more messages Processing...
    1. hobekyo
      hobekyo
      • 11 years ago

      I was a bit curious about the fact that Co doesn't contribute to the model so a tested multivariable linear regression. However my R code gave the answer Residuals:
      Min 1Q Median 3Q Max
      -15.1672 -2.8807 0.7144 2.8338 8.0371

      Coefficients:
      Estimate Std. Error t value Pr(>|t|)
      (Intercept) 1.120e+00 5.867e+00 0.191 0.8490
      Pr -4.753e-05 1.644e-03 -0.029 0.9770
      Pe 5.134e+00 7.820e-01 6.565 1.31e-09 ***
      Ca 3.046e+00 1.277e+00 2.385 0.0186 *
      Co 1.656e+00 7.928e-01 2.089 0.0388 *
      No 6.934e+00 5.916e-01 11.722 < 2e-16 ***.

      • 11 years ago
    2. hobekyo
      hobekyo
      • 11 years ago

      The matlab result was exactly the same. It seems that there is correlation between Co and the final score.

      • 11 years ago
  • carowjp
    Contest Holder
    • 11 years ago

    By saying 'must outperform a linear regression' I mean do NOT submit a linear regression (it will be rejected). I am looking for less naive approaches.

    • 11 years ago
  • cgullapalli
    cgullapalli
    • 11 years ago

    Would like to know more about the project. How can i do so?

    • 11 years ago
    View 2 more messages Processing...
    1. cgullapalli
      cgullapalli
      • 11 years ago

      what is the time limitation for submission.

      • 11 years ago
    2. carowjp
      Contest Holder
      • 11 years ago

      The contest is active for 3 days.

      • 11 years ago
  • deepakbansal91
    deepakbansal91
    • 11 years ago

    before starting work on this project....i would like to talk to you. I have done many projects on regression analysis.

    • 11 years ago
  • aminbros
    aminbros
    • 11 years ago

    hi, let me do your project.............................

    • 11 years ago
  • Danielharman
    Danielharman
    • 11 years ago

    I have seen the data. I am ready for it.

    • 11 years ago

Show more comments

How to get started with contests

  • Post your contest

    Post Your Contest Quick and easy

  • Get tons of entries

    Get Tons of Entries From around the world

  • Award the best entry

    Award the best entry Download the files - Easy!

Post a Contest Now or Join us Today!