Scikit-LLM Turns Custom Entity Definitions into a Zero-Shot NER Pipeline

Custom named entity recognition often means collecting labeled examples and tuning a model to fit a particular vocabulary. Scikit-LLM offers a lighter way to try the task: define the entity categories in plain language, send text to a compatible large language model, and receive candidate extractions. In the example here, that setup identifies executives, companies, products and cities in three short technology business sentences, without training a model from scratch.

That makes the approach useful for prototyping or small extraction jobs where a labeled dataset is unavailable. It also shifts the work: instead of preparing training labels, teams need to write precise category descriptions, manage model access and decide how to handle the model’s output. The demonstration is small, so it shows how to assemble a pipeline, not how reliably it will perform across a large or varied corpus.

Configure a compatible model endpoint

The notebook runs in Google Colab and routes Scikit-LLM requests through Groq’s API. The example requires a Groq account and API key, stored in Colab Secrets under the name GROQ_API_KEY. It installs the library with:

!pip install scikit-llm -q

Scikit-LLM follows OpenAI API conventions. Its configuration can point to a different compatible service by setting a custom base URL, then supplying the key retrieved from Colab:

from google.colab import userdata
from skllm.config import SKLLMConfig

groq_key = userdata.get('GROQ_API_KEY')

SKLLMConfig.set_gpt_url("https://api.groq.com/openai/v1")
SKLLMConfig.set_openai_key(groq_key)

This setup makes the provider endpoint a configuration choice, while the NER code stays within Scikit-LLM’s estimator interface. The article describes the selected Groq option as compatible with a free tier; that can lower the barrier to trying the notebook, but availability and terms depend on the provider. The example does not report latency, usage limits, or a cost comparison.

Describe the entities, then run extraction

The central input is an entity dictionary. Each key names a category to extract, and its value explains what should count. Those descriptions guide the model’s decisions, taking the place of a conventional set of labeled training examples in this workflow.

entities = {
    "EXECUTIVE": "The full name of a company leader or individual.",
    "CORPORATION": "The name of a technology company or business.",
    "PRODUCT": "A specific hardware device, software, or technology feature.",
    "CITY": "A geographical location or municipality."
}

X_unstructured = [
    "Satya Nadella announced Copilot integrations for Microsoft at the summit in Seattle.",
    "Elon Musk supervised the rollout of the Cybertruck at the Texas facility.",
    "Jensen Huang showcased the new Blackwell architecture for Nvidia in San Jose."
]

For these definitions, the intended results include people such as Satya Nadella and Jensen Huang, products such as Copilot and Blackwell architecture, and locations such as Seattle and San Jose. The choice of wording matters: “CITY” includes municipalities, while the example sentence mentioning a Texas facility names a state. The model returns no CITY extraction for that sentence, a result consistent with the category description. If a workflow needs states as well, it should define that category explicitly.

The task-specific estimator is GPTExplainableNER. The article identifies openai/gpt-oss-20b as a compatible model for this configuration at the time it was written. After passing the entity descriptions and model identifier to the estimator, fit_transform() processes the input texts:

from skllm.models.gpt.tagging.ner import GPTExplainableNER

ner = GPTExplainableNER(
    entities=entities,
    model="custom_url::openai/gpt-oss-20b"
)

extracted_data = ner.fit_transform(X_unstructured)

The code keeps the familiar fit-and-transform pattern, but the example does not train a new model. It calls a pre-trained model to interpret the descriptions and text, then returns extraction results. That distinction matters for adoption: there is little setup compared with a fine-tuning workflow, but the quality of each result depends on model behavior and the instructions supplied.

Expect raw output that needs handling

The initial results aren’t consistently returned as ready-to-use dictionaries. In the example, outputs appear as strings containing tags such as <entity>, <tag> and <value>, alongside explanatory reasoning and spans marked as non-entities. A first display helper attempts to convert string representations of dictionaries with ast.literal_eval; if that fails, it prints the raw result. That fallback is useful for inspection, but it doesn’t turn the tagged text into structured records.

A second helper uses regular expressions to find each entity block and extract its tag and value. It collects values into lists under category names, producing a cleaner view of the three examples:

• The Nadella sentence yields EXECUTIVE: Satya Nadella; PRODUCT: Copilot; CORPORATION: Microsoft; and CITY: Seattle.
• The Musk sentence yields EXECUTIVE: Elon Musk and PRODUCT: Cybertruck.
• The Huang sentence yields EXECUTIVE: Jensen Huang; PRODUCT: Blackwell architecture; CORPORATION: Nvidia; and CITY: San Jose.

This post-processing is a practical part of the pipeline, not cosmetic cleanup. The regex helper expects the model to return particular tags and does not include broader validation for malformed or unexpected output. A production workflow would need to check parsed values, handle missing categories and decide what to do when the response format changes. The example’s raw reasoning text also adds material that the final entity dictionary doesn’t need, so downstream systems should consume the parsed values rather than treat the full response as a stable data format.

What the demonstration establishes

Across three short sentences, the model identifies the intended people, products and companies, and it recognizes the two city names. That’s a useful proof of concept: domain-specific categories can be expressed directly, and an LLM can apply them without a custom labeled dataset. But three handpicked examples don’t measure precision, recall, consistency, or performance on ambiguous names and longer documents. The article provides no benchmark comparison with conventional NER libraries.

For teams evaluating the method, the next step is to test representative text against a reviewed set of expected extractions, then inspect both missed entities and incorrect matches. Category definitions should also reflect the actual schema: a broad description such as “individual” may include people who aren’t executives, while a narrow category such as CITY will leave out other geographic types.

Scikit-LLM makes this kind of experiment concise, and its estimator interface can fit into a Python machine learning workflow. The tradeoff is that model output still needs parsing and verification, while provider access and model compatibility shape deployment. The [Scikit-LLM project](https://github.com/iryna-kondr/scikit-llm) and [Groq API key page](https://console.groq.com/keys) are linked in the source for readers who want to reproduce the setup.

Iván Palomares Carrascosa writes, speaks and advises on AI, machine learning, deep learning and large language models.