As the name suggests, SLMs are smaller versions of LLMs with fewer parameters. Some of them can run even on personal computers, laptops, or mobile devices. Instead of depending entirely on remote cloud systems, many SLMs can operate offline or within an organization’s own infrastructure.
In this article, we examine what SLMs are and why they may become relevant for Civil Society Organizations (CSOs) that work on data-sensitive projects or operate with limited resources. We present several relevant SLMs and provide practical guidance on how they can be used in practice. By understanding the potential and limitations of SLMs, CSOs can better assess where these models may offer safer, more transparent, and more economical alternatives to LLMs.
Why SLMs Matter for Civil Society Organizations
SLMs may provide significant value for CSOs by helping to address several key concerns.
1. Data Privacy and Protection of Vulnerable Communities
Many CSOs handle highly sensitive information such as testimonies, human rights documentation, legal case files, or personal data of activists or marginalized populations. Using LLM APIs can create a difficult trade-off between efficiency and confidentiality. Uploading documents to external servers may expose organizations to data leaks, surveillance, or unauthorized access.
SLMs can address this issue because they can be deployed locally on organizational servers or even laptops without sending data to third-party providers. This enables document summarization, translation, classification, or transcription while maintaining greater control over sensitive information.
2. Infrastructure and Connectivity Challenges
In many regions, particularly across parts of Africa, Latin America, and rural areas, internet connectivity remains expensive, unstable or inaccessible. LLM APIs require internet access. This can cause an operational fragility during outages, large uploads, or bring additional costs.
Some SLMs can operate offline once installed locally. This allows organizations to continue working even under poor connectivity conditions. This offline capability can be particularly valuable during crises or emergencies (presented in Table 1 below).
3. Financial Sustainability
Most CSOs operate with limited and unpredictable funding. Subscription-based LLM services or high API usage costs can quickly become unsustainable.
SLMs can reduce this barrier because many are open-source, free and less computationally demanding. Although there may still be setup and maintenance costs, organizations can avoid recurring monthly fees tied to commercial AI platforms.
4. Surveillance and Political Risk
Organizations working in politically hostile environments often face surveillance threats. Dependence on international platforms may introduce risks such as government data requests, platform censorship, monitoring activist networks.
Using SLMs will help organizations reduce their digital dependency.
5. Ethical and Environmental Concerns
Popular LLMs such as ChatGPT are not transparent about their training data and data processing steps. Their development and operation also require substantial computational resources, which can contribute to a significant environmental footprint.
SLMs offer an alternative model that may better align with principles of decentralization and community ownership since they are usually open and more transparent than closed LLMs. In terms of environmental impact, although SLMs still consume computational resources, their operational footprint is generally smaller than LLMs.
Examples of SLMs
Table 1: presents an overview of some popular SLMs that could be used to perform several CSO tasks, along with their providers, supported languages, specializations, additional features, and context lengths.
General Purpose: Qwen3-0.6B, Phi-4-mini-instruct, Llama-3.2-3B-Instruct, Gemma-2B, tiny-aya-global are general-purpose SLMs. They can perform many text analysis tasks such as entity recognition, summarization, or question and answering. Qwen3-0.6B and Phi-4-mini-instruct can also support tasks that are more complex than text generation tasks and require reasoning such as comparing information and fact-checking.
Languages: Qwen3-0.6B or Phi-4-mini-instruct support multiple languages. This means they might understand prompts in other languages and execute requested tasks in a prompt in the target language. However, this does not mean they are limited strictly to supported languages. If the target language is similar to languages included in training, the models may still be able to respond reasonably well in that language.
Translation and Transcription: Models such as Translategemma-4b-it and Voxtral-Mini-4B are specialized models for particular tasks such as translation and transcription, since they were fine-tuned on task-specific training datasets.
Long-context document analysis: Phi-4-mini-instruct, Llama-3.2-3B-Instruct and Ministral-3-3B-Instruct support long context windows. This can be useful when working with longer documents such as policies and reports. For CSOs, this may support tasks such as summarizing long documents or answering questions from a report.
Multimodal Tasks: Some SLMs can process images in addition to text. For example, Ministral-3-3B-Instruct and Llama-3.2-3B-Instruct have vision capabilities, while TranslateGemma can support translation from text and image inputs. Such models may be useful when organizations need to analyze screenshots, scanned documents, diagrams, or text embedded in images.
Coding: Some SLMs such as CodeGemma and Qwen2.5-Coder-3B can support coding tasks such as code generation, debugging, SQL/SPARQL query writing. For CSOs, this can be useful for accessing information from structured data sources such as Wikidata.
How can new SLMs be tracked?
If you are new to SLMs and are unsure which model to choose, it is often best to start with a widely used and well-supported model before exploring others. Additionally, if you plan to write prompts in a language other than English, it is important to verify whether the SLM was trained on that language or on linguistically similar languages, as multilingual support and performance can vary significantly between models.
Image: It is easy to track new, popular or specialized SLMs on HuggingFace.
GenAI is evolving rapidly, with new models and tools emerging almost every day. As a result, keeping up with the latest developments and trends can be challenging. Nevertheless, the following sources are useful for tracking recent advancements in SLMs.
Huggingface: has a large collection of open-source AI models, including SLMs. As shown in the image above, you can search for models using specific parameters (i.e., <4B can show SLMs), and sort them based on their popularity or date of creation.
GitHub: another platform that hosts many code repositories and projects. You can search for usages of SLMs or repositories such as SLMs-Survey can provide additional knowledge about SLMs.
Academic Sources: ACL Anthology and ACM Digital Library are also valuable references for finding applied research papers that utilize SLMs in journalism and the social sciences. For example, in this article, the researchers examine SLMs for news summarization.
Limitations
Table 2: presents a comparison between SLMs and LLM APIs in terms of internet access, data privacy, cost structure, performance, ease of setup, and scalability.
Despite their advantages in terms of cost-efficiency and privacy, SLMs also have several limitations. Compared to LLM APIs, SLMs may underperform on highly complex tasks that require advanced or multi-step reasoning, for example, document fact-checking or detailed analysis. They may also have shorter context windows, making it more difficult to process long prompts containing extensive instructions or large inputs, such as multi-page documents. In addition, self-hosting or adapting SLMs for specific use cases may require technical expertise and suitable local hardware.
How to Use SLMs in Practice
For Users with Less Technical Knowledge:
These tools provide user-friendly interfaces for running and interacting with local models without requiring extensive setup or programming knowledge.
For Users with More Technical Knowledge:
These tools offer greater flexibility, customization, and performance optimization for deploying and experimenting with SLMs in development environments.
For Advanced or Purely Technical Use Cases:
These frameworks are designed for developers who want to build AI-powered applications, agentic workflows, and custom pipelines using models programmatically. Alternatively, model pages on Hugging Face sometimes provide example code for running the model.
Methods for Improved Results in SLMs
Several practical methods can help better the SLM performance and response quality.
Break complex tasks into smaller, easier subtasks
Instead of asking the model to solve a large problem all at once, divide the task into sequential steps. For example, fact-checking can be separated into identifying the claim, gathering sources, evaluating evidence, and summarizing conclusions.
Use detailed prompts, assign roles
Precise instructions help reduce ambiguity and improve output quality. Including the desired format, tone, or objective often leads to more reliable responses than vague requests. For example, instead of simply asking a model to do a task, you can write: “You are a fact-checker. Use critical thinking to evaluate this claim, identify reliable sources, and explain whether the claim is accurate.”
Provide examples or context when possible
Supplying background information or sample outputs helps the model better understand the task. For example, if you would like to define atomic facts extraction for fact-checking, you can add an example to the prompt, such as one in the table below:
Table 3: an example of providing examples and context, using the information about Techsoup's Digital Activism Program.
Combine SLMs with retrieval tools or external knowledge sources when needed
Since smaller models may have limited knowledge or context windows, connecting them to documents, databases, or search systems can improve factual accuracy and domain-specific performance.
Use agentic frameworks for multi-step workflows
Frameworks such as LangChain, LangGraph, or Smolagents can coordinate steps in a workflow, tool usage and memory across tasks. These frameworks also enable multiple SLMs to collaborate on a single complex task by assigning different subtasks to specialized models. For example, in a video verification workflow, an agentic pipeline could use Voxtral to generate a transcript, use Llama to translate the transcript into another language, and then call a web-search agent to gather and evaluate supporting evidence from the internet.
Final Thoughts
SLMs are not a complete replacement for LLMs, but they offer an important alternative for organizations that prioritize privacy, affordability, transparency, and control. For CSOs operating in resource-constrained or politically sensitive environments, SLMs may provide a more sustainable and ethical approach to adopting GenAI applications.
For CSOs, the value of SLMs is not only technical, but also strategic. They can help organizations experiment with GenAI while keeping sensitive data within their own infrastructure, reducing dependency on commercial platforms and adapting tools to local needs, languages, and contexts. Additionally, learnings from CSOs can also inform the development of SLMs themselves due to a variety of CSO tasks.
At the same time, it is important to remember that both SLMs and LLMs can generate inaccurate or entirely made-up information, aka hallucinations. Outputs should always be verified through expert review and trusted sources, especially when used for decision-making, advocacy, or sensitive operations.
Your Feedback Matters
What did you think of this text? Take 30 seconds to share your feedback and help us create meaningful content for civil society!
Disclaimers
This piece of resources has been created as part of the AI for Social Change project within TechSoup's Digital Activism Program, with support from Google.org.
AI tools are evolving rapidly, and while we do our best to ensure the validity of the content we provide, sometimes some elements may no longer be up to date. If you notice that a piece of information is outdated, please let us know at content@techsoup.org.
The content was created, reviewed, and edited by Ipek Baris Schlicht with AI assistance.