April 27, 2024 MDG

LLM’s From Skill to Proficiency 

Early on, LLMs had a reputation for laziness, often stopping short of a complete answer or list. But that final 10% is crucial – it’s the difference between ‘just okay’ and truly effective, between a rough idea and a tool you can actually use

Skill: A developed ability to do something well.
Knowledge: Theoretical or practical understanding of a subject.
Aptitude: A natural ability or talent for acquiring skills and knowledge within a domain.
Mastery: Comprehensive or in-depth skill and knowledge in a particular area, indicating a high level of proficiency.
Proficiency: The level of competence in a skill or area of knowledge.

We have good performance in complex language understanding tasks, language translation, customer support automation and dare I say LLM’s excel at creative content generation.  That said, I am still finding it hard to say we have continuous proficiency 

Highlighting what’s missing in an “almost” answer, LLM’s call it a feature and it can help us learn or prompt us to think about something in new ways, this applies well to creative applications but not so much for the other proficiencies listed above. 

The unspoken truth right now is that LLMs cannot provide 100% proficiency at any task consistently.  It is very hard to guarantee complete accuracy or reliability.  LLM’s are probabilistic not authoritative.  Factual accuracy, nuanced context and originality in reasoning are  all less then ideal as they are remixes and representing existing information. 

I believe that LLM’s will remain tools to annotate and augment human domain expertise but never replace it.  There will be improvements through fine tuning and continuous learning and the eventual combining of reasoning tools to better achieve factual accuracy.