News about a tea competition results awhile back started me considering what a tea competition evaluation and scoring process might be like. As my mind works it would be interesting to compare some of the methodology and evaluation processes to the very different but vaguely related process of judging teas against personal preference when drinking them for enjoyment. I didn’t expect a lot of telling overlap to emerge but did eventually get around to researching the subject.
The real thing.
Overview
Obvious enough, but evaluation for a tea competition would involve a few basic steps:
- a consistent brewing methodology for evaluation, under carefully defined and controlled conditions
- a set of review criteria, categories against which the tea would be evaluated (taste, color, body / mouth-feel, etc.)
- a scoring system to enable judging which teas are evaluated as superior (although it seems conceivable that a tiered pass-through process could replace numerical scoring)
Really there would be a lot more to it, related to all sorts of controls of conditions, example integrity controls, methodology to ensure consistent judging related to throwing out some “outlier” results, and so on. I suppose this is a good place to mention that I have absolutely no experience with such a process, which is why researching this subject appealed to me. The prior categories and functions might benefit from restatement from someone with more experience and training, but that was one goal, to consult and summarize research sources.
I attempted to consult “tea experts” in one place such a request would make sense, in a tea professionals oriented LinkedIn group, but that didn’t get far, so hopefully some gaps get flagged once this is published instead (the Cunningham’s law idea).
The ISO standard
If there is such a thing this must wrap it all up, right? Not so much. ISO 3103:1980 does cover how to brew tea consistently for evaluation, though, described as such:
The method consists in extracting of soluble substances in dried tea leaf, containing in a porcelain or earthenware pot, by means of freshly boiling water, pouring of the liquor into a white porcelain or earthenware bowl, examination of the organoleptic properties of the infused leaf, and of the liquor with or without milk or both.
It is headed in the right direction, but the Wikipedia excerpt–a sample of partial detail summary, mind you–shows why it just didn’t get there (also noted by the Tea with Gary blog awhile back):
The pot should be white porcelain or glazed earthenware and have a partly serrated edge. It should have a lid that fits loosely inside the pot…
If a small pot is used, it should hold a maximum of 150 ml (±4 ml) and must weigh 118 g (±10 g).
2 grams of tea (measured to ±2% accuracy) per 100 ml boiling water is placed into the pot.
Freshly boiling water is poured into the pot to within 4–6 mm of the brim. Allow 20 seconds for water to cool.
The water should be similar to the drinking water where the tea will be consumed.
Brewing time is six minutes….
If the test involves milk, then it is added before pouring the infused tea.
Milk added after the pouring of tea is best tasted when the liquid is between 65 – 80 °C.
It’s not as bad as it looks. One possible way to go is brewing all the teas exactly in the same way and tasting around the broad difference in how personal preference brewing parameters would vary, and my understanding is that this is still essentially a common approach (maybe just not always using this set).
Tasting using milk might seem odd but I have it on good authority that some prominent tea purchasers do exactly that–judge teas in an altered final consumption form. A tea mentor (of sorts) once suggested that it’s best to drink tea cooler to taste it, around 60° C, and I’m not certain about any optimum but of course he is right in principle (per my own experience, at least).
More developed standard references
There must be a lot more, but Google did identify a few for a starting point.
The World of Tea site reference actually includes the ISO 3103 process as one alternative, and suggests using both that methodology and a second review based on more conventional brewing parameters, partially defined as such:
| Tea Type | Temperature | Time for First Steeping |
| Green Tea | 160F/ C | 1min (Yellow and White teas omitted) |
| Oolong Tea | 190F/ 88C | 1.5min |
| Black Tea | 205F/ 96C | 1min |
The article goes on to suggest evaluating the following criteria, along with reviewing leaf appearance:
1.What color is the liquor?
- What does the liquor smell like? (How does it change from steep to steep?)
- What does the liquor taste like? (How does it change from steep to steep?)
- How many steepings can the tea withstand and still produce acceptable flavor?
So far so good; reasonable brewing and evaluation guidelines. I’d hoped for a bit more on scoring (maybe some summary of the rating process), but regardless of what a reference set up would be, it would be hard to really improve on a subjective overall evaluation that compared teas directly. Direct evaluation of limited samples might just be limited when attempting to judge a large number of teas, or to combine multiple judgments from different judges.
This also seems like a good place for an aside about how limiting the number of factors evaluated would also limit the completeness. For example, that methodology suggested review against liquor color, liquor smell, taste, and the number of steepings: four factors (and a good start). A more general Tea Vicious reference on how to evaluate teas suggests some others (only partially cited, with their section on methodology omitted entirely):
- How does the inside of the mug lid smell?
- Does the aroma linger? Which part of it lingers longer? Which shorter?
- Does any part of the aroma remind you of another substance?
- Does any part of the taste come first and some other later?
- What do the different parts of the taste remind you of?
- Which part of your tongue gives you the sensation of the taste?
- How does the taste stay?
- Does the taste change during the process of tasting?
- What is the texture of the tea liquor?
That would be way too many criteria for judging a lot of teas against each other but the general concepts of feel and finish (aftertaste) might be retained.
Maybe it’s just as well to cite a bit of two other references, and then wrap this up. One from Tearroir provided an example of the type of scoring system I’d expected:
Each tea receives a base of 60 points, and we award each tea up to 40 points based on the following categories:
Color and appearance of tea pre-brew: 5 points
Aroma: 5 points
Flavor and Mid Palate: 10 points
Mouthfeel and Finish: 10 Points
Overall quality level: 10 points
The engineer side of me loves this approach. Based on how I evaluate teas myself, related only to preference, I’m not so convinced a lot wouldn’t drop out in trying to weight groupings like these though, that it might fail to fully capture how a few strengths or weaknesses can really determine impression. I just tried a Hong Shui tea with an amazing range of great flavor characteristics and a slight sourness issue that corresponded to how the feel was presented, so even the break-down might not be so simple.
Lastly, the North American Tea Championship, related to the World Tea News / Academy / Expo, provides another good example of evaluation methodology, which they summarize as such:
Each submission into the class is evaluated blind and through organoleptic analysis of the following characteristics: dry leaf, brewed color, brewed aroma, brewed flavor, brewed mouth-feel and brewed harmony. An overall numerical value on a 100-point scale is then calculated based on the ratings of each characteristic above.
Nice! They just don’t mention how the actual scoring is calculated, but they did include a fascinating description of how they make judging more consistent by throwing out inconsistent / outlier scores (and you really should read that if you happen to love statistical sampling process). A sample of brewing parameters makes it clear they go with brewing methodology closer to ordinary brewing:
| Category | Type | Temp | Minutes | Weight | Special |
| Assam | Origin Authentic | Boil | 5 | 2.5g | |
| Breakfast Blend | Origin Authentic | Boil | 5 | 2.5g | |
| Bai Hao/ Oriental Beauty | Breakfast Blend | Boil | 2/4 | 2.5g | Steep open bowl; 2 infusions; 3 cupping bowls per tea |
| Chai | Masala Chai | Boil | 5 | 3g | Add 20% warm milk/sugar mix |
That partial citation of “Signature Famous Teas” doesn’t do parameter choices review justice, but it’s probably enough to add that green teas are cited as between 175 and 180, depending on type. Interesting to also note World of Tea went down to 160 on their guidelines for green tea brewing; make of this difference what you will.
That last reference didn’t really seem intended to give the full details of the methodology and evaluation process but it’s nice they passed on that much.
Conclusion
I had intended to circle this back more to a comparison against how people drink and evaluate teas for enjoyment but it seems as well not to do much with that. It’s just different.
It’s interesting to consider how flaws in tea might factor into these approaches, or how individual positive aspects might be rated versus a balance of aspects in a tea, but it also seems as well I that don’t muddle the review here too much with my own speculation. Besides, with more expert input or research about this much content, I could always write up a “part two” post and ramble on in that one. And scoring systems never went far in these references; lots more room to add to that.
I clearly see the engineer in you John. I’ve always been curious about tea tasting. I remember being invited by Steve Smith into his back room one day when he was doing a tea tasting. He had lined a huge wooden table with 30+ white cups of tea. He stood in front of each, took his silver tasting spoon, slurped the tea into his mouth, moved it around and spit it out into a bowl. He moved along quite quickly, without comment or paper. I was floored when he identified his favorites and asked for my opinion.
I think the tasting a typical tea drinker would want to do is worlds apart from what the professionals do when tasting hundreds of tea a day. Hard to imagine drinking that much so spitting it out makes sense.
I heard Steve Smith once broke the record for number of teas sampled in one day. The number was in the hundreds, high hundreds, if I recall correctly. After you learn what a particular tea should taste like, it’s very easy to choose between them. However, to me, the ultimate is if it makes me smile, sometimes even laugh. Like finding a needle in a haystack.
Love that imagine – a sip and a smile!
I also studied as a philosopher, which is closer to being an engineer than one might expect. All the same I’m of two minds when I drink tea, and can appreciate analysis and comparison but in some ways feel this really does conflict with just enjoying the tea. Related to that one perspective I’m interested in the idea of scoring teas, and the use of evaluation categories, but part of me would also reject any system as truly capturing the essence of the experience. I guess some internal complexity can be nice. At least I don’t end up arguing with myself about all of it.