Practice UKCAT is WRONG

[Linnaeus]

This is a letter drafted but never sent. Basically, aside from all the other reasons why the UKCAT is useless, some of the answers to the full practice test seemed to be wrong (I could be taking crazy pills, of course). Here's the ranting letter detailing which questions I think are wrong or ambiguous:

Dear

RE: The UKCAT Practice Paper – Why some questions are ambiguous or outright wrong

I can only begin to imagine the difficulty in separating the wheat from the chaff during medical school admissions – what is an admissions officer to do when faced with a debased exam system in which thousands upon thousands of applicants score maximum marks? I do not claim to offer a panacea for your admission woes, but I do write to say that, to me at least, the UKCAT feels like an unnecessarily blunt instrument for selection.

However, this is not the thrust of my letter; I appreciate that the UKCAT is a hurdle that I, just like everyone else, must traverse if my medical aspirations are to be realised. So, I recently took the full practice paper available on the UKCAT website in preparation for my upcoming exam. I am not writing to question the efficacy of this exam (although I would not be the first if I did so: insert sBMJ Ref); my concerns are more specific and immediate.

To elaborate: Upon checking the answer pdfs provided on the UKCAT website I was horrified to find that a number of questions were apparently ambiguous, and that some of the answers appear to be outright wrong. I have no issue with a test being hard – how else are you to avoid ceiling effects? – but I take umbrage when I do not think I am being assessed fairly.

Here are the particular instances I would appreciate you inspected:


Verbal reasoning

9.
The answer should be ‘CAN’T TELL’ rather than ‘TRUE’ because there is no data given of the reasons for piercing. While a new start may be far more likely than any other for prompting someone to get pierced, the passage does not explain how many alternate reasons exist for getting pierced. For example, if the likelihood of a new start prompting piercing was 10% relative to 1% for all the other, say, 90 reasons, it would be fair to say that it is by far the most common reason, as the passage states. However, this would NOT mean one could then deduce that, upon meeting someone with a piercing, it is ‘more than likely’ a new start prompted their piercing; for this to be true the passage would have to state that the likelihood of a ‘new-start piercing’ was 50% or more. It does not.

Quantitative reasoning

1-4.
The figure used for questions 1-4 isn’t so much wrong as misleading. The passage states that the chart represents the mid-prices between week 1 and week 5, but it then states that vertical lines pertain to the Monday of each week. Therefore, what the chart really shows are the values between the opening Monday of week 1 and the opening Monday of week 6. This causes interpretation of the graph to be slower and may lead to incorrect answers (e.g. in Q4.).

12.
The questions asks “by what percentage... does the cheapest price... undercut the price of the most expensive...” This leads to the following ambiguity: should the £125 difference be measured against the cheapest or the most expensive price? The ‘correct’ answer in the pdf is ‘from the cheapest’ but I would argue that one should assume the undercutting is being done to the higher price by the lesser one. Thus, it would be fair to say that the lesser price is ‘16% cheaper,’ rather than saying that the higher priced one is ‘18% more expensive;’ the latter is not akin to ‘undercutting.’ Seeing as both 16 and 18% are possible answers, the candidate is left none the wiser about which option to choose and the ‘correct’ answer would seem to be the less intuitive of the two.

14-15.
The passage gives the following example of how a 25-year-old earning £8,000 would be taxed in 2006-7: “Have a tax-free allowance of £5,035; pay tax at 10% on £2,150; and pay tax at 22% on the rest [my bold]” – it is this last piece of information which is misleading. If one looks at the working shown in the answers pdf file, tax is not worked out in this additive style. For example, if someone earns £41,000 (as in question 14) then I agree that £5,225 of this would be deductible in 2007-8, leaving behind £35,775. I also agree that £2,230 of this is then taxed at 10%; so far so good. Where I disagree with the working is in the next step. The passage states that 22% is paid on the rest. To my mind this would mean that the tax-free £5,225 and the £2,230 upon which 10% has just been paid would no longer be taxed. Therefore, what would be taxed at 22% would be the remaining £33,545 (= 41,000 – 5,225 – 2,230), which fits into the £2,231-34,600 band without crossing the cut-off point of £34,600. However, this is not how it is done in the working. Instead £32,370 (= 34,600 – 2,230) is taxed at 22%. One cannot infer from the information given that this is necessarily the way to calculate the tax. In fact, I would argue that the example involving £8,000 people is worded poorly and is likely to cause errors in what is essentially a series of exceedingly simple calculations, masking a candidate’s true arithmetic prowess.

23.
The question talks about ‘quiet country roads’ which one might reasonably assume are ‘extra urban.’ However, it becomes clear once one has done the calculation that the answer one derives is not listed among the options. Thus, one is then forced to repeat the calculation with a different value (in this case, one must induce that ‘combined’ is the appropriate measure) but in so doing valuable time is lost. When time is already in such short supply, this feels unfair.

31.
Where on earth do the ‘fuel increase’ figures cited in the answers pdf come from?!

Decision analysis

I realise these answers are more subjective and open to debate, but I feel the absolute nature of answering questions 24 and 25 misses some alternative answers:

24.
‘Conditions’ is deemed unnecessary despite the fact that there is nothing that approximates to it. I would argue that weather could be adequately captured by something like: F(G35, 7GCD3, G3101, GBD)

25.
‘Value’ is described as being akin to positive (which I do not think it necessarily is, which is why I feel a term for value would be helpful) and is therefore deemed redundant. In contrast, ‘society’ is deemed as useful, but I would argue that it could be captured by something as simple as F2.


I have listed up to 12 things I spotted which I think are either wrong, ambiguous or verging on unfair. Out of the total 175 items posed in the UKCAT, I feel that 7% is too high when so much is riding on one exam and a single percentage point could make all the difference. I realise this is a specimen paper, but how am I to know that the ‘real thing’ will not be similarly blighted?

My GCSEs and A-levels are good, I have a first class biological science degree and PhD from Cambridge, not to mention a great deal of hard-won health related experience, so the notion that my academic past and potential could be distilled into one 90-minute exam (ignoring the personality test) is both exasperating and rather laughable. Why is this any more fair than merely using a person’s first degree mark? Of course, this too would be rather crude and it potentially disadvantages ‘late developers,’ but at least it is both a measure of one’s academic ability and one’s commitment across three years of study, rather than a trite 90-minute vignette. I won’t even begin to list the problems and mismanagement encountered by last year’s cohort (see Student BMJ...) – I think you get my point.

Finally, I worry that this letter may damage my chances of gaining a place on your esteemed course, which is still something I dearly, dearly want to do; from a pragmatic point of view, I probably should have kept quiet and protected my self-interests. After all, this letter is hardly going to alter your admissions policy. However, I feel very strongly about the issues of fairness and access – not to mention good, old-fashioned common sense – and I felt compelled to write and expose this ‘UKCAT factor’ for what I think it is: something which must surely hinder more than it helps; something which creates a multiple choice admissions lottery rather than level the playing field.



Yours sincerely,

[SupraSpinatous]

send it big man... right now!

[iheartmedicine]

Your letter is compelling and well-articulated. I would not advise you to send that letter until you are accepted on to a course, however. There is no need for you to be a martyr at this point!

[Linnaeus]

Cheers guys. I kind of wimped out and have decided to send it if I get an offer; self-preservation and all that

[iheartmedicine]

I think that that is a prudent decision. You do not want to do anything to hinder your chances at this point. I would be annoyed if I were an admission's tutor and I received that letter. They have carefully thought about how they use the test, so it is best not to question their authority at this point!

[SupraSpinatous]

it may help your app actually.
If you're right then they can't really argue the point!

[Zedd]

I'd be tempted to send it now, if you're totally right then it'll show balls and brains.

I'm glad I'm not the only one that finds the answers confusing when looking at the passages/info and the questions. Everytime I've done the practice I've found myself asking how the hell they came up with the answers, I wish that they would explain their reasoning so that I could understand where I'd gone wrong if at all.

[iheartmedicine]

I think that some of the answers on the verbal section are contentious to say the least.

[Doc.Oc]

Bring that letter to your interview haha. And yes, the practice paper really is bull.

[Nerrep]

It's probably a good thing you haven't read 'Passing the UKCAT and BMAT', or you might have actually exploded.

To be entirely fair, I don't think that 8 (generally minor) mistakes over a ~200 question paper is that terrible, especially given that UKCAT are unlikely to spend a huge amount of time (money) on checking and reviewing practice questions that nobody is ever going to actually face in an exam (and your QR score of 890 further supports this by indicating that you were getting marks for what you perceived as being correct in the real thing).

It's also certainly not true that finding a small number of mistakes reflects on the usefulness of UKCAT as a test of functional intelligence.