Aspose.LLM 26.11.0

Aspose.LLM for .NET

Aspose.LLM for .NET is a local LLM inference library for C# and .NET. Use it to run large language models on your own machine or server, build chat-driven workflows, send image input with vision presets, and keep prompts, model weights, and responses inside your own process instead of calling a hosted inference service.

The SDK targets netstandard2.0, works across modern .NET runtimes, and uses built-in presets for model families such as Qwen, Gemma, Llama, Phi, DeepSeek, and vision-capable variants. Native llama.cpp runtimes for CPU, CUDA, HIP, Metal, and Vulkan are downloaded automatically on first use.

No built-in LLM is included in Aspose.LLM. A preset is configuration only: you choose which open-source model to use, and the model file is obtained separately on your own machine, under the model publisher's license, not the Aspose license. Model names are used solely to identify the models, with no affiliation or endorsement implied; each name remains the property of its owner.

Why Aspose.LLM for .NET

• Run local LLM inference from .NET applications on CPU or GPU.
• Start stateful chat sessions with message history and session persistence.
• Use built-in text and vision presets instead of hand-building inference pipelines.
• Send text-only or multimodal prompts through one high-level AsposeLLMApi.
• Tune context, sampling, acceleration, and runtime behavior through parameter bags.
• Bring your own GGUF model by extending PresetCoreBase or overriding preset settings.

How It Works

1. Choose a preset for a text or vision model family.
2. Create a single AsposeLLMApi instance for the current process.
3. Let the SDK resolve native llama.cpp runtime assets for your platform.
4. Send prompts through SendMessageAsync, optionally with image bytes for multimodal requests.
5. Persist or reuse chat sessions when you need multi-turn conversations.

Supported Environment

Install from NuGet

dotnet add package Aspose.LLM

Apply a license before calling chat APIs. Aspose.LLM does not run inference in evaluation mode, so a temporary or commercial license is required for StartNewChatAsync, SendMessageAsync, and SendMessageToSessionAsync.

Choose a Preset

• Qwen25Preset for general text chat and local assistant workflows.
• Qwen3Preset for text reasoning and chat-oriented workloads.
• Qwen25VL3BPreset for image + text multimodal prompts.
• Qwen3VL2BPreset for larger-context vision scenarios.
• Gemma3VisionPreset for vision-enabled Gemma workflows.
• CPU variants such as Qwen25_3BPresetCpu or Llm32_1BPresetCpu when GPU acceleration is unavailable.

Text Chat Example

using Aspose.LLM;
using Aspose.LLM.Abstractions.Parameters.Presets;

var license = new License();
license.SetLicense("Aspose.LLM.lic");

using var api = AsposeLLMApi.Create(new Qwen25Preset());

string response = await api.SendMessageAsync(
    "Explain retrieval-augmented generation in two sentences.");

Console.WriteLine(response);

Multimodal Vision Example

using Aspose.LLM;
using Aspose.LLM.Abstractions.Parameters.Presets;
using System.IO;

var license = new License();
license.SetLicense(@"C:\licenses\Aspose.LLM.lic");

using var api = AsposeLLMApi.Create(new Qwen25VL3BPreset());

byte[] imageBytes = File.ReadAllBytes(@"C:\images\sample.png");

string response = await api.SendMessageAsync(
    "Describe what is shown in the image.",
    media: new[] { imageBytes });

Console.WriteLine(response);

Session Lifecycle and Memory Cleanup

SendMessageAsync is conversation-oriented. The first call creates a chat session when no session is active; later calls reuse that active session and append to its history. The session is not closed automatically.

For independent one-shot prompts, close the session when the response is no longer needed:

string sessionId = await api.StartNewChatAsync();
string response = await api.SendMessageToSessionAsync(sessionId, prompt);
api.CloseChatSession(sessionId);

ForceCacheCleanup(...) prunes the KV cache inside the active session, but it does not destroy the session's native context. Use CloseChatSession(sessionId) to release per-session native context memory, and dispose AsposeLLMApi when the loaded model itself is no longer needed.

Control Cache Directories

When a preset leaves cache paths at their built-in defaults, the SDK can take the cache roots from environment variables:

• ASPOSE_LLM_MODEL_CACHE_DIR
• ASPOSE_LLM_RUNTIME_CACHE_DIR

Precedence is:

1. preset path
2. environment variable
3. built-in default

Built-in defaults are under the local application data directory:

• models: Aspose.LLM/models
• runtimes: Aspose.LLM/runtimes

Load Presets From JSON

AsposeLLMApi.LoadPresetFromJson(...) loads one preset from one inline JSON document. The JSON must contain extends, which selects the built-in preset used as the base. Any other provided sections override that base preset.

var preset = AsposeLLMApi.LoadPresetFromJson("""
{
  "extends": "Qwen25Preset",
  "contextParameters": {
    "contextSize": 8192
  },
  "samplerParameters": {
    "temperature": 0.2
  }
}
""");

using var api = AsposeLLMApi.Create(preset);

AsposeLLMApi.SavePresetToJson(...) exports a built-in preset back to the same contract, so a user can tweak a working preset in code and then export an exact JSON example:

var preset = new Qwen25Preset();
preset.ContextParameters.ContextSize = 8192;
preset.SamplerParameters.Temperature = 0.2f;

string json = AsposeLLMApi.SavePresetToJson(preset);
Console.WriteLine(json);

If you want a full preset snapshot instead of only overrides, pass includeDefaults: true:

string fullJson = AsposeLLMApi.SavePresetToJson(preset, includeDefaults: true);
Console.WriteLine(fullJson);

Customize a Preset

using Aspose.LLM.Abstractions.Parameters.Presets;

var preset = new Qwen25Preset();
preset.BinaryManagerParameters.ReleaseTag = "b10988";
preset.ContextParameters.ContextSize = 8192;

using var api = AsposeLLMApi.Create(preset);

Use this pattern when you want to keep the model-family defaults but adjust runtime tag, context window, batching, or acceleration preferences for a specific deployment.

Common Use Cases

• Local AI assistants and chatbots for .NET applications.
• Document automation, classification, and question answering workflows.
• Code generation and code analysis scenarios.
• Multimodal vision Q&A with text and images.
• Offline or controlled-deployment AI integrations where data should stay local.

Before First Run

• Apply a temporary or commercial license before starting a chat session.
• Expect an automatic native runtime download on the first launch for the selected backend.
• Plan local disk space for runtime assets, model weights, and chat-session state if you persist sessions.
• Keep your first validation scenario small, then move to larger presets or larger context windows.

Important Notes

• First use downloads the matching native llama.cpp runtime for your platform and acceleration backend.
• AsposeLLMApi allows only one active instance per process at a time.
• Presets are the recommended entry point for model selection, sampler defaults, context size, and acceleration behavior.
• Vision support uses the same API surface as text chat and adds media input through the media parameter.

Resources

• Documentation
• API Reference
• Supported Presets
• Supported Acceleration
• Licensing
• Temporary License
• NuGet Package
• Free Support Forum
• Paid Support Helpdesk