Insights into UPSC Prelims
A Statistical Justification: Why you should attempt more Qs in UPSC Prelims
This article draws some critical observations from a statistical study of the results of our Prelims Test Series, 2019. The study is based on test result samples from our offline batch since the offline format resembles the actual testing environment more strongly than the online tests. We have chosen the last few tests (Tests 38, 40 and 42 – General Studies), near completion of the test series, to eliminate any biases arising from a general lack of preparation in aspirants.
In Civil Services (Prelims) examination, the only two numerical factors that determine your success are: a) your attempt rate, i.e. number of questions attempted, and b) your accurate rate, i.e. a percentage of right answers against total attempts. A strategy to increase your overall score based on these two numerical factors is crucial, yet quite neglected in the strategy of most aspirants, perhaps because amassing knowledge assumes more importance than an exam-centric strategic planning.
Following, there are only two ways you can crack the exam: either you have a high accuracy rate and you attempt a satisfactory number of questions, or you have a moderate accuracy rate and attempt over 90% of the paper! If you have an accuracy rate below 60% (i.e. only 60 questions right out of 100 attempted), even if you attempt 100 questions, you will end up with only 94 marks, which is below the general category cutoff of 98 in Prelims 2018. This article makes a statistical case for why you should attempt maximum number of questions, if you belong to a certain category or cohort of aspirants (those who have a stable accuracy rate in any Prelims mock tests that you may have previously attended; this is made clear later in the article).
Consider the following line graphs drawn from our test result data:
Horizontal axis: Rank in the test; Vertical Axis: a) Accuracy rate or percentage (Orange), b) Number of unattempted or unattended questions (blue) for fear of negative marking or lack of knowledge.
Note: We choose 50% accuracy as the benchmark accuracy for our graphs (rather than 60% as shown earlier), because Insights tests are a bit on the tougher side and accuracy is usually lower than in the actual exam. Add 10% to the accurate rate for UPSC exams since the guesswork is slightly easier there improving accuracy rates. We neglect the data showing accuracy rates below 50% since it is not significant for our discussion, i.e. these aspirants would not be able to cross cut off. A part of data sample used for drawing these graphs has been shared at the end of this article.
Observe the following trends from the data:
- a) the relationship between the rank and accuracy rate
- b) the average accuracy rate for a certain cohort or rank group (e.g. 1-20, 20-50)
- c) the relationship between the accuracy rate and number of unattempted questions
SECTION A: As for (a), the relationship between the rank and accuracy rate, you will note that there is a gradual, but slow decline in the accuracy rate in the rank list. This means that high rankers tend to have a higher accuracy rate than low rankers. This observation is perhaps not so surprising, and seems logical. The next two observations are rather surprising and form the statistical basis for our arguments in this article.
SECTION B: As for (b), the average accuracy rate for a certain rank group, you might tend to believe that different toppers may have very different accuracy rates, and some get good marks by attempting more number of questions, and some by having a higher accuracy rate. This would seem to suggest you that accuracy rates must vary wildly from person to person in a rank group. The argument that then follows is: each of us should attempt only those questions of which we are completely sure, and leave the rest to avoid negative marking. While it sounds ideal, it is far from truth. This is made clear in the following section.
SECTION C: As clear from the graphs, the accuracy rate seems fairly stable across the ranks and does not seem to depend on the number of attempted questions. This means that you are not likely to have a higher accuracy rate only for the reason that you have marked less number of questions, or that you are not likely to have a lower accuracy rate merely because you are marking more number of questions. In other words, there is no evidence from the test results that shows that marking more number of questions reduces the accuracy rates and thus hurts your chances of selection; rather the opposite. Based on a broad trend of accuracy rates, we draw the following table from Tests 38-42 results:
|Rank Range||Average Accuracy Rate|
(add 10% additional, since accuracy improves in UPSC examination)
|10-100||Upper scale: 70%|
|10-100||Lower scale: 65%|
|Below 200||40-55%, statistically insignificant since they won’t qualify|
We believe that offline rankers between 100-200 form a strong representational model for an average UPSC aspirant. With this assumption, our data supports the view that most candidates are likely to have an average accuracy rate of about 55-65% (i.e. 65-75% in UPSC), and this does not depend on their number of unattempted questions. This is an important observation for the following reason:
Considering that you are an average candidate (in terms of your rank in test series), you would actually be better off attempting more questions than less. The lower your accuracy rate, the more questions you will need to attempt in order to score higher. With an average accuracy rate (70% UPSC, 60% Insights), someone who attempts only 80 questions with 56 Qs right (in UPSC) and 24 wrong, gets a score of 98, exactly borderline! Surprising right? This is precisely where a large pool of candidates get stuck, since 80 questions seems to be the norm, as many dread attempting any further. But, as we have just shown, since accuracy rates remain largely stable for serious aspirants, and do not necessarily depend on the number of attempts, marking more questions makes sense. As the data shows, more attempts increases your chances significantly. Someone who attempts 90 questions, with the same accuracy rate, surprisingly gets a higher score of about 108, well above cutoff. Note that he did not do anything extraordinary to achieve this higher score; same accuracy rate, yet different attempt rates bring two different results!
A common argument against such an approach is that, attempting more number of questions reduces your accuracy rate, but this could not be farther from the truth. This is probably an illusion most of the aspirants harbor. Our statistical analysis (section C) shows that accuracy rates in most aspirants belonging to the same group (Top 100, between 100-200) are remarkably similar. What this suggests is that merely a few serious aspirants may mark questions based on pure guesswork, the majority does it pragmatically. Had that not been the case, with more number of attempted questions, we would have noticed a wild fluctuation in the accuracy rates, but we know this is not true. Instead, what the data shows that aspirants are successful either because they know their subject and have high accuracy rate, or they are good at increasing the number of attempted questions. Several comments on our test discussion forums testify the same, where a candidate would claim to score a 120 totally out of the blue, whereas others might struggle getting even 90 marks! This implies that you should not worry too much about marking more questions, as long as you are not doing a wild guesswork.
Another branch of the aforementioned argument (against marking questions) is about hypothetical certainty. At least in theory, one may say that, if you know only 50 questions, and know them with absolute certainty, why would you mark the rest of the 50 questions hurting your accuracy rate and the chances of selection? While theoretically valid, this argument loses out on empirical grounds for several reasons; one of which is that most aspirants know some questions with certainty but are unsure of a number of other questions. A neat and clear hypothetical certainty about the answers is rare and therefore statistically insignificant. It does not bear relevance to a diverse set of real world testing results. There will always be some 30-40 questions in the exam that you will not know with cent percent confidence, an intelligent guesswork (click here to read related article) that increases your number of attempts will serve you best in this situation.
This is not to say that you should always end up marking 100 questions, but to suggest you the following: Monitor your accuracy rate in the tests you have written so far, especially the last 5-6 tests where you are more likely to have gathered broad knowledge and would have been able to score better. Having gathered this information, you should devise your marking strategy based on your accuracy rates. If your accurate rate has been consistent or stable around 60-65%, go ahead and mark more number of questions. But, if your accuracy rate has been erratic, suppose 40% in one test and 80% in another, and then falls back to 50%, then it is a difficult choice to make and you need to be careful with your number of attempts; it may backfire. But, if the tests are standardized and if you write them seriously, avoiding pure guesswork, then taking a chance by marking even slightly uncertain question will do no harm.
The key take away from this article for you is to understand your preparation more concretely and employ this to your advantage even if you feel you are not the best of the lot. Note down your past performances in any mocks tests you might have written, check how many mistakes you have made and categorize those mistakes (such as silly mistakes, poor interpretation, lack of information, conceptual flaws etc.). If you come to know that you make the same mistakes again and again, despite your efforts to improve, calculate your accuracy rate and then see if you can crack UPSC Prelims with this accuracy rate. If you have consistently had a lower accuracy rate, it is time that you practice more and more mocks to improve your guesswork so that you increase your attempts, which is crucial other than all the revision you are doing. There is a limit to how much you can know, and given the erratic pattern of this exam, it is not merely knowledge that determines one’s success or failure. Often the difference between a successful and not-successful candidate is everything but knowledge. Work on improving your exam performance as much as revision and consolidation.
Some of the assumptions we have made in this analysis are that, a) the chosen cohort is an ideal one, and b) the level of preparation of the subscribers of the test series reflects that of the broader population of aspirants. These assumptions are pretty standard, and do not go against the general conclusion drawn from the data.
Generic advices like “mark more questions” may have been given in the past to you, but as we have access to more information, we, here at Insights, are able to make better researched decisions in terms of guiding students.
Institutions with access to more data, thus, have a great resource since it brings greater awareness in terms of making choices and allows them to constantly reflect and improve on their choices. An advice tendered sometime back may or may not be applicable in a dynamic context, especially when the exam pattern and the expectations of UPSC change every 2-3 years. Fresh empirical evidence, therefore, stands crucial to present day decision-making.
With a pan-India subscriber base and access to the results of several thousands of aspirants, at various levels of preparation, we have a strong representative population of aspirants, which allows us to offer suggestions not only based on experience but also on hard data: all this with a single pointed objective of simplifying the exam preparation. Good luck!
Note: Below, we are sharing excerpts from the data sample (Test 40 offline, ranks 70-99) used for this study (Names and roll number have been kept confidential). Co-relating this data with what we have discussed above should hopefully give you some insights!