Judging inconsistencies - I improve my car then get destroyed.

Thats of course a good idea Pascal, I do intend to have a conversation, electronically if not in person, with those in the command chain for that show and those clubs. While I am still a relative concours newbie, after two seasons and 6 shows I was pretty well convinced something was wrong with the scoring at that show.

It would seem to me that perhaps we need some kind of scoring oversight board. A committe that just reviews scores, compares scores of the same cars from one show to the next and flags obvious problems such as the one experienced at this San Luis Obispo show.

As it stands now I have no reason to go back to that show , thats for sure, so I guess it would be in the best interests of the club President and show hosts to address these issues...

Daniel Arsenault
Lakewood, CA
1994 XJ12 Morocco Red
1995 XJS 4.0 Rose Bronze

i think you really need to discuss this with the chief judge of the club who docked you 24 points vs 3 to 5 at the other shows... i can't imagine any chief judge not interested about such discrepancies. your 20 point discrepancy is clearly an anomaly

one of the issues we have in judging at least in some clubs, is a lack of debriefing after the annual concours. I believe clubs should be encouraged to hold a post judging session, reviewing judging and deductions to see what should have been done differently.

We had our concours this past week end and although it went fairly well, there were some calls, some decision made, some oversight maybe that we should discuss among judges. it would be a great learning experience.

I wonder if this is not something that should be either required or at least recommended in the rule book. It woudl help.

when I jugded, I clearly point out what I was deducting. as long as the owner shows a genuine interest in impoving his car, and it there is no discussing / arguing, I think it is the right thing to do.

One thing that is being worked on, in addition to improving judge training (the JCRC has been spending a lot of effort on that part) is improving reporting. As of now, clubs are supposed to file a judge report listing who judged what but this information is either not submitted or simply filed away and only used in case of a formal protest. Chief judges currently have very little data on judges and while some of the most experienced CJ know his judges, it's harder for less experienced CJs. in the near future, judge reports will be filed on line and we will be building a database of judging data as well as judges quilfications/certifications. This might help spotting recuring out of whack scores and maybe address the issue with these judges.

Pascal

Complain to the head judge of the concour, the councours chairman and the club president where you got screwed. It's pretty easy to do, show them all your other score sheets and they'll speak for themselves.

If the club leadership has any brains, they'll do something about it (ie "reinstruct" the judge or not allow him to judge again). If the club leadership does nothing (essentially condoing this type of behavior) then word will get out and people who concours their cars will avoid that club's future concours.

If anyone is still monitoring the thread...

I have now received all my score sheets from this years 4 Concours. Scores were 9.966, 9.955, 9.900 and 9.755.

The 9.900 had a mathematical error which will result in it being corrected to a 9.907, which will raise position at that show from 5th to 4th but does not significantly affect the overall standings.

My main concern with the wild variation in scoring was to see how it was possible for such variation to happen. In looking at the 9.755 , which was a ridiculously low score I could only attribute to poor judging, I noted that the judging team cut vast swaths of excessive point deductions. Areas I had never before been deducted for were taken off, areas I had been deducted for only slightly, they hacked off 3, 4 or even 5 entire points. Areas I know I had no deductions , they took off. What makes the judging for the show even worse was the fact that it was raining and no exterior judging took place. I shudder to think how ridiculously low my score would have been if the exterior had been judged as well. I remember it raining much of this show, perhaps these judges just didnt want to be out in the rain?

Just to put things into perspective. I have been to 6 Concours in two years, with the car largely identically prepared each time. My interior deductions have been 4.3 pts, 1.8 pts, 5.1 pts, .4 pts, 3.3 pts, and at the final show a WHOPPING 24.5 pts!!!

I have browsed a number of classes, Concours and looked at scores from across the country. I cannot find another example of such blatantly poor judging in any other class or show. Look at the three cars who were judged by this team at this show:

A car that scored 9.966 earlier in the year in sunny weather now scores 9.755. Another car scoring 9.990 earlier in the year now scores 9.805? And yet another car scoring 9.990 scored a 9.747 at this show.

Someone needs to look into the judging team responsible for this anomoly.

Daniel Arsenault
Lakewood, CA
1994 XJ12 Morocco Red
1995 XJS 4.0 Rose Bronze

It seems as though the concours results posting online can sometimes be delayed up to a month also, although again, I do not know why. I have not personally had to enter scores using the online form but it doesnt seem to me like it should take a month. It makes the online standings a lot less "real time".

Again as to the judging consistency issue, what it means basically is that the national class winners win only partially on the quality of their cars. The rest comes from them being lucky enough not to get a judge who takes off whole points rather than tenths. From what I have seen you can take a car in to 5 concours identically prepared and see such a wide variance in scores that it then comes all down to luck. Did you get lucky and get 5 "easy" judges, or did you get unlucky and get 5 "tough" judges. Whether or not there are areas in the country which have a higher concentration of "easy" or "tough" judges I cannot say, I just think we should not have either.

We should not be letting pure random chance and luck determine our national class winners. The cars should speak for themselves. I am comfortable saying this, I say this knowing I do not have a class championship contending car, and that my car will always be in the middle of the pack at best.

Daniel Arsenault
Lakewood, CA
1994 XJ12 Morocco Red
1995 XJS 4.0 Rose Bronze

>>>.

Bob you are correct I have never attended your show but can I remind you of the conversation you and I had just this last weekend when you told me that there were a few disgruntled people at your concours this year that thought the scoring was too tough. I talk to people all the time like I did with you and I normally take the time to listen to what they have to say. I learn that way. In addition to what I have been told, the scoring results are all on the JCNA site for all to see. What more do I need to know.

So let me clarify my position. I am not saying or attempting to suggest that your club or even Ottawa, Toronto or Indianapolis are doing anything incorrectly. They could very easily be just a few of those that are doing it right and everybody else wrong. It is just not being done consistent across all the clubs right now. Like it or not, these four clubs tend to come out on the low side of the scoring sheets when it comes to qualifying or attempting to qualify for a National or Regional awards. This is exactly what I told you face to face last Friday night Bob you never disputed it them.

I am not trying to offend anyone by bringing this to everbodies attention, least of all you Bob. However there is inequity in the way we are doing this right now and the results talk for themselves. The indisputable evidence of what I am saying can clearly be seen in the published results. Do the maths, take any entrant that has attended any three of these shows and there are a few of them and see where they would stand on those results alone.

Bob.
92 V-12-VDP Black Cherry #39
92 V-12-VDP Oyster #90
87 V-12-VDP
86 XJ6-Soveriegn
85 XJ6-VD

Gee Bob, No doubt you'll correct me if I'm wrong, but going through my records it appears that you've never attended a show in Michigan so, if that's the case, you have no first hand knowledge of how we judge. The great majority of our judges have been with me 10 or more years and are pretty knowledgable of the cars as many of them judge the same class year after year. While we may not be perfect it's not because we don't try or care.
To compare the average score of, say, all C5 cars at one show against the same in another doesn't mean a thing unless it's the same cars. Can judging vary from show to show, absolutley as there is no way that I know of to have every judge see every deduction the same way. We all are aware that the system is not perfect but most Chief judges that I personally know take their task very seriously as do, I'd like to think, the great majority of judges. Sure there are times when deductions can seem to be excessive but on the other hand when judges miss things, which probably happens more then the above, no one points out the missed deducts! We've showed cars at least 60 times over the past 20 years and long ago learned to take the good with the bad and just enjoy the friends that we've made at these shows!
Concerning recuiting judges the day of the show, the Michigan club has been forced to recruit non club members to judge on only one occasion in the 17 years that I've held this position. Back around 90 or 91 the show was held at the Edsel Ford House and we were expecting about 80 cars and ended up judging 116!

I've always said that if each entrant judged his/her own car, and were totally honest, all scores would drop and 100 point cars would be a thing of the past! Maybe that's what we should do, have each entrant turn in a list of all defects and if the judges find something else the car is disqualified! Any takers?

Bob Stevenson Chief Judge JAG OF MICH.

what I meant is that it is currently up to the clubs, more than JCNA, to do something about it.

Pascal. I agree with everything you wrote with the exception of the above statement. Each of your suggestions would be an enormous step forward if implemented and enforced. However I believe it is JCNA's responsibility to enforce the rules by insisting on adoption without exception by each and every club. Official sanctioning should be refuses to any club that fails to comply. I would like to add one thing I believe you left out that I really feel we require the individual breakdown sheets for each model of car.

I suggested the part about Chief Judges becoming ombudsmen to mediate scoring disputes, just as a humorous side bar. However thinking about it, it would certainly have the effect of encouraging the chief judges to keep a tighter reign on his or her individual judges, so as to prevent them from being at the show site till late in the night.

Bob.
92 V-12-VDP Black Cherry #39
92 V-12-VDP Oyster #90
87 V-12-VDP
86 XJ6-Soveriegn
85 XJ6-VD

Bob,

what I meant is that it is currently up to the clubs, more than JCNA, to do something about it.

Two types of problems have been raised. Rules (judges qualifications, procedures, etc...) and lack of consistency.

1)- Rules...

The rules are already there... judges are required to attend a seminar, pass a test, judges at least twice in 3 years, stay up to date with rule book changes and re test every 3 years. Some clubs follow these rules... some don't. I know we (SFJC) do and I also know that we don't draft anyone on concours morning... if we use judges from neighboring clubs, they are well known, certified, long time judges.

What we need is to make sure clubs follow these rules... and here are a few ideas that woudl help.

- have chief judges send in their Judge Reports whcih they are currently obligated to fill out but not submit. These could easily be done online to make it easier, to and make the info available.
- Judge Tests should be sent in to JCNA. this woudl send a message that JCNA is serious about judge training and keeping track of who is a certified judge. Right now, Chief judges don't even report who passed the test and attended the seminar... according to the rule book, they simply need to notify their membership chair who in turn will include this info in the annual roster update... up to a year later! have the tests sent in to JCNA, update the roster with the letter J and year of certification. Ideally send in an updated member card... this way, when a judge offers to help at a concours, the chief judge knows he is certified and when... Certification will also be available on the website thru the frequent JCNA nr updates
- get the regional directors involved... if a problem arise which doens't necessarialy warrant a protest, members shoudl advise their regional directors who in turn could discuss the issue with the chief judge, club pres, concours chair, to make sure rules and procedures were followed. Regional Directors could also advise other clubs of potential issues to make sure they do not happen again. things like what was mentioend in another thread... a judge touching a car without permission or kneeling on the seat... missing something obvious, etc...

I think these woudl help improving the process with little effort.

2)- discrepancies...
i don't think it's feasible to have the Chief judge taking the role of an arbitrator during a concours...this will turn out into the poor guy working till 8pm trying to resolve issues...

More training and more guidelines are what we need there... Is there enough in the rule book to teach new judges what to deduct? I wonder.

ex... let's assume you find 5 little stone chips (1mm) on the bonnet of E-type and one side is not aligned. Gap at the cowl is good but the junction is off at the wheel arch. what are the official guidelines for deductions? some will say to deduct 5 times the minimum 0.1 for the chips... I don't see the answer in the rule book... and I remember feeling a bit lost the first time I jduged... Poor fit deduct can range from 0.2 to 8 points. what do you deduct for that spot assuming all the other panels have the proper gaps? 0.5? 1 ?... 2 ??? I don't see any guidelines anywhere. one judge might deduct 0.5, another one 2 points. Over an entire car, it can easily add up to 20,30 points or more.

same with cleanliness... 100% subjective... no guidelines... couple of dirt spots on a carpet... small...what do you do from 0.1 to 12 ? I 'm sure that 3 different trained judges will come up with 3 diferent deducts...

one way to better train judges would be thru precise examples, with pictures and official recomendations... in print or online... but we need to better train new judges. this is key to improve concours judging and also to get new judges, another critical problem.

Pascal Gademer
South Florida Jaguar Club
72 E-type 2+2
00 XKR Coupe
99 XJR

Daniel... clubs are supposed to mail the sheets within 2 weeks... some don't... if that's the case, contact the chief judge or concours chair and ask for the sheets. same with posting... it shoudl be done wihtin 3 weeks but some clubs take forever (always the same...)

when that's the case, contact the club and ask about the sheet.

or why not... go to their club page and post your request in their club forum. even if there isn't much traffic, maybe a public request will help.

The rules are already in place.. some clubs choose to ignore them. see my reply to Bill re judge training... same problem. the rules are there but some clubs ignore them...

Pascal Gademer
South Florida Jaguar Club
72 E-type 2+2
00 XKR Coupe
99 XJR

Ill let everyone know when I get the score sheet. I still haven't receieved the scoresheet for a concours on Sept 25th so it could be a month before that for the Oct 17th show comes in.

I am the first to admit I am still feeling my way through the Concours experience, Ive been competing now two years and have participated in 6 sanctioned events. Thus far what I have seen is some very poorly trained judges, at least one instance of favoritism, and wide variances in scoring with respect to both my own car, several others cars, and with respect to the scores posted nationally for the class I compete in.

It has me concerned enough to volunteer to train as a judge, and I am considering sponsoring or assisting in writing a model guide for D07/ C13 since it appears as though the level of knowledge of at least this class is just not up to par for whatever reason. I have not given up trying to help the situation and much of my posts lately have been more venting frustration than anything.

Daniel Arsenault
Lakewood, CA
1994 XJ12 Morocco Red
1995 XJS 4.0 Rose Bronze

>>>>>>I agree that there are consistency issues in judging and unfortunately, not much can be done there

Do you not mean, not much will be done there? If JCNA had the will there are lots that can be done about it.

For example Chief judges of each individual club have the means to set where the bar is placed for their team of judges. Right now it is all over the place because they are not setting the bar the same. Consistency can be achieved if every chief judge is made to understand the level of judging that JCNA insists on maintaining. In addition, to facilitate that some form of consistency is achieved, it should be made mandatory for each club to hold a judging seminar each year for the judges they intend to use in order to qualify for JCNA sanctioning. Sample judging can be conducted allowing directives for judging standards to be given directly to the judges. Clubs that use imported judges from other clubs should require that those judges they choose to use, produce confirmation that they have attended at least one judgeÆs seminar that year. This would go part way to remedy the problem of many Judges operating without a working knowledge of the rulebook and in some cases absolutely no knowledge of the vehicles they are supposedly judging.

This time honored event of recruiting judges the morning of the show should be thrown out with the trash. Concours chairpersons should be required to co-ordinate with the chief judge to organize these things ahead of time. You would not expect to see them hunting for a location the morning of the show. Why then is it acceptable that they do this with the judges. A list of judges should be required to be submitted along with the location and date when sanctioning is requested. Sure this is going to take more organization on the part of the clubs but so does preparing a car for concours.

We all work hard and spend lots of money to prepare our vehicles and we pay the entry feeÆs expecting to be judged by knowledgeable people and people that have at the very least a basic understanding of the rulebook. A lot of the times we are getting neither. A vast number of these judges have been recruited the morning of the show and have never even opened a rulebook. The results are, like a lot of other peoples the scores for my cars are all over the place. I have a range from a low of 9.937 to a high of 9.997 and thatÆs a difference for ô60ö deduction points over the five clubs I have attended this year.

In fact if you have any aspirations for running for a national title there are four clubs in my area you donÆt want to attend. For whatever reason, across the board the marks they award appear to be a lot lower than the average. They are in order Ottawa, Indianapolis, Toronto and Michigan. I made the mistake this year of attending three of them.

Of course another option for solving this problem would be to allow the entrant to see his scores before they are registered. With the chief judges given the additional roll of arbitrator to settle disputes.

Bob.
92 V-12-VDP Black Cherry #39
92 V-12-VDP Oyster #90
87 V-12-VDP
86 XJ6-Soveriegn
85 XJ6-VD

I agree that there are consistency issues in judging and unfortunately, not much can be done there...

Judges should know the rules and be familiar with them. but if they don't, then what? your comparison with baseball umpires doens't work since JCNA judges are not paid but volunteers ... how do you want to reprimand them? it can be only done in case of serious issues involving clear violations of concours rules. For inconsitencies between events, there is not much taht can be done. In any case, it doesn't change how cars are ranked at one show...

when you get the score sheet for this last concours, it woudl be interesting if you could post a follow up wit the detailed deductions so we can see where teh differences are...

Pascal Gademer
South Florida Jaguar Club
72 E-type 2+2
00 XKR Coupe
99 XJR

I thought the situation was bad before this weekend, then I showed up at the joint JOCLA and SF Jag club show and all 3 cars in class, all scored in the 9.7s. One of these cars had a 9.998 earlier this year in bright sun, this event was held in the rain covering many minor blemishes. This truly boggles my mind.

I am more convinced than ever that we have a juding problem. There obviously is insufficient training and no adherable standard, or we would not have these kinds of ridiculous score swings. This is not minor score variation attributable to difference in judge, this is completely out of whack. I was told by another contestant that this particular judge " just doesnt believe in taking off tenths".

To say my car can get scores ranging from 9.981 to 9.755 is absolutely ridicuous and I think we have a HUGE problem. It might be enough to stop my attendance at concours and at 30 years of age I am the type of person that JCNA needs attending events!

Perhaps we need a national review system for judges performance. They are like umpires in Major League Baseball. When an ump has a bad game, the footage is reviewed by the commissioner and reprimands are issued, and it is possible for an umpire to be fired. Ive now done 6 sanctioned events in the past two years , and I was planning on campaigning both XJS and XJ12 next year, but with these wild score swings, I wonder if I will continue....

Daniel Arsenault
Lakewood, CA
1994 XJ12 Morocco Red
1995 XJS 4.0 Rose Bronze

The solution is simple, avoid clubs conocurs that use judges with attitudes like this. In my case that means I won't enter my car in my own club's concour from now on. Oh and my club (Jaguar Club of Southern New England) wants me to continue to do the scoring for the concours? Well, I've got news for them, they can find somebody else to do it from now on. Our club's Concours Chairman was the judge at the JANE concours who trashed my car.

And the Jaguar Club of Southern New England wonders why participation in their concours has been steadily declining over the past five to ten years. Keep up the good work JCSNE, pretty soon no one will want to enter cars in your concours.

Ive come to the conclusion there may be nothing you can do about poor judging, but I'd like to try anyway. I've done 5 shows in a row with scores in D07 ranging from 9.935 in bright sun to 9.981 in a shady are with tree pollen covering some of the imperfections in my car. In all cases the car was essentially identically prepared, only the ambient conditions varied. It seemed like my car settled in at around a 9.955 which I think is a fair assessment of the condition of the car.

I recently did one new local club's inaugural show and competed in a 5 car field of very well prepared cars, two of which I have competed against before this season. All cars scored at least 30 to 50 hundredths lower than at any show I have been to this year. My score for this show was a 9.900 , 35 points lower than my worst previous score, and one which I do not believe fairly represents my car.

I was very dissapointed, not only for myself, but for the other 4 competitors who I think were badly judged at this show. I felt the order of cars was more or less correct, but the entire field was artificially low. I come to this conclusion because it seems to me that for the one judge team in this case to have been right, 4 other judging teams were wrong. Seems more likely that this one team was wrong.

I also feel bad for the club, as the experience makes me very unlikely to attend their next concours, Im sad to say. At this point I am just hoping not to run into this judge/ judge team again anytime soon.

As to the comments that we should expect poor judging. Well, I'm sorry, but preparing for a concours is a lot of work and " the contestant should expect to be screwed one week...." is just not acceptable!!! I cant imagine why anyone who thinks it is would do anything more than display class. We should try to do the best job we can on judging , adhering to the rules, being consistent. Im going to put up rather than shut up by training as a judge for the classes I know best, D07 and later C13 and XJS as I go down the road.

Can we do a better job? Yes. I've already experienced favoritism, judges missing key non authentic items in my class. Judge inexperience ( hey are those picnic tables original??? YES this question was asked by a judge once!!!) Improvement is definitely needed across the board, and I hope to take a more active role in promoting this improvement as next season opens.

Daniel Arsenault
Lakewood, CA
1994 XJ12 Morocco Red
1995 XJS 4.0 Rose Bronze

I have now determined who the judge was and on the contrary, he was a very experienced judge.

It seems pretty clear to me that my case isn't a matter of an inadequately trained judge but there's something going on here that is just plain not right.

It starts with the fact the HE DIDN'T EVEN LOOK AT THE TOOL KIT AND TOOK DEDUCTIONS then goes downhill from there and there must be more to the story and if and when I find out, I'll update you.

I think Pascal's advice to contact the club's (JANE) head judge with my situation is the right way to go and since this particular judge was from my own club (JCSNE), I should also inform our head judge of this judge's performance and hopefully he will either be "reinstructed" or not allowed to judge in the future.

These huge discrepancies are way beyond the variations (human factors) you'd get from one judge to another.......

I don't mind "bad calls" but not when they are made deliberately or with malicious intent.

Ouch, that hurts !

Well, the only way around this that I can see.....and I think it has been mentioned before...is for the judges to have your previous deductions as they judge your car. JCNA would have to have them in a data file for downloading by judges before each event. This way the judge *today* will see what was deducted the last time and can determine if you corrected the fault. Of course, this doesn't solve the entire problem...the judge today can still find new problems the the other guy missed last week. And sometimes we won't agree with what they find....

Next, also mentioned before, would be centralized judges training. As it stands, each club trains their own judges and, this being the case, lack of uniformity is a given.

Add to this the concours rulebook itself. Scoring would be more consistant if the rules/deductions were more specific.

Then we have the undeniable human element....never to be totally overcome regardless of what we do.

Hate to say this but, really, its all part of the game. As mentioned a few weeks ago the softball players usually keep quiet when the umpire makes a "bad call" against their team......knowing full well that the next "bad call" may very well be in their favor.

Doug Dwyer
JDRC/NWA
1987 XJ6 III
1988 XJS V12 Coupe

I would contact the club's head judge and advise him of the discrepancies.

he obvioulsy can't change your score but he should be interested to look at the discrepancy and address them at their next training session.

it's always going to be a problem because of the nature of judging. I woudln't necessarily say the the judge didn't take his job seriously, maybe he was a new judge.. i don't know... what I do know, from a club point of view, is that having qualified judges is difficult. Few members are willing to spend the time needed, both training and judging.

Pascal Gademer
South Florida Jaguar Club
72 E-type 2+2
00 XKR Coupe
99 XJR