VeeFriends.WikiImporter 5.3.2

VeeFriends.WikiImporter 📘🌍

VeeFriends.WikiImporter is a .NET library designed to automate the import and transformation of character data from ClickUp into a structured format, incorporating AI-driven enhancements and media assets. The library integrates with multiple services, including Google Drive, Cloudinary, and ChatGPT, to enrich character data with link previews, images, and more.

Features

• Automated import of character data from ClickUp
• AI-driven content processing via ChatGPT
• Integration with Google Drive for media assets
• Cloudinary support for optimized image storage
• Link preview generation for social and media content
• Caching and error handling for efficient processing
• Dependency Injection support for seamless integration

Installation

Install the VeeFriends.WikiImporter NuGet package in your project:

dotnet add package VeeFriends.WikiImporter

Usage

CLI: Medium Blog Preview

Generate the VeeFriends WebPageModel JSON for a Medium/blog URL without writing to Cosmos. The CLI uses the full blog importer path, including SEO generation and Cloudinary asset upload/reuse:

VF_MEDIUM_API_KEY=... \
OPENAI_API_KEY=... \
CLOUDINARY_CLOUD_NAME=... \
CLOUDINARY_API_KEY=... \
CLOUDINARY_API_SECRET=... \
LINKPREVIEW_API_KEY=... \
dotnet run --project VeeFriends.WikiImporter.Cli -- \
  blog import-url "https://blog.veefriends.com/example-title-abc123def456" \
  --out out/blog-page.json

The CLI accepts a Medium URL, blog.veefriends.com URL, or raw 12-character Medium article ID. It reads Medium, generates SEO, ensures Cloudinary-backed assets exist, and writes JSON locally. LinkPreview config is required because the production importer registers the social preview service, even when the specific blog article may not use it. Cosmos comparison and upsert should happen in the calling workflow after an explicit dry run and approval. Slack importer notifications are disabled by default for CLI runs; pass --send-slack only when needed.

1. Configuration

Configure the necessary services in your Startup.cs or Program.cs file:

using Microsoft.Extensions.DependencyInjection;
using VeeFriends.WikiImporter;

var services = new ServiceCollection();
services.AddWikiImporter(
    configureChatGpt: options => { options.ApiKey = "your-api-key"; options.Model = "gpt-4"; },
    configureSystemPrompt: options => { options.Prompt = "Your system prompt here"; },
    configureClickUp: options => { options.ApiKey = "your-clickup-api-key"; },
    configureCloudinary: options => { options.CloudName = "your-cloudinary-name"; },
    configureGoogleDrive: options => { options.ApiKey = "your-google-drive-api-key"; },
    linkPreviewOptions: options => { options.ApiKey = "your-link-preview-api-key"; },
    configureDZineAi: options => { options.ApiKey = "your-dzine-ai-api-key"; }
);

2. Executing an Import

Inject the CharacterImporter service and call ExecuteImport to start importing character data:

public class WikiImportService
{
    private readonly CharacterImporter _characterImporter;

    public WikiImportService(CharacterImporter characterImporter)
    {
        _characterImporter = characterImporter;
    }

    public async Task ImportCharactersAsync()
    {
        var importOptions = new ImportOptions
        {
            OnProgress = (progress, _) => Console.WriteLine($"Progress: {progress}%"),
            OnError = (error, _) => Console.WriteLine($"Error: {error.Message}"),
            DelayBetweenRequests = TimeSpan.FromSeconds(1),
            SkipCache = false
        };

        await _characterImporter.ExecuteImport(importOptions, CancellationToken.None);
    }
}

3. Error Handling

The library provides structured error handling using logging and event-driven error reporting:

try
{
    await _characterImporter.ExecuteImport(importOptions, CancellationToken.None);
}
catch (Exception ex)
{
    Console.WriteLine($"Import failed: {ex.Message}");
}

Extending the Library

Adding a New Data Source

To extend the importer with a new data source:

1. Implement a new data provider service that retrieves content.
2. Modify CharacterImporter to include the new source.
3. Integrate processing logic using ModifyWithGoogleDrive or ModifyWithLinkPreview patterns.
4. Register the new service in ServiceCollectionExtensions.

Dependencies

VeeFriends.WikiImporter relies on several external services:

• ClickUp: For retrieving character wiki pages
• ChatGPT: For AI-driven text processing
• Google Drive: For media file storage and retrieval
• Cloudinary: For optimized image hosting
• Link Preview: For generating link previews of media content
• Microsoft.Extensions.DependencyInjection: For dependency injection support

License

This project is licensed under the MIT License. See the LICENSE file for more details.

Credentials for importer tests and storage

Provide provider credentials through environment variables or a managed secret store. The test Startup reads OPENAI_API_KEY, CLICKUP_WORKSPACE_ID, CLICKUP_USER_PRIVATE_KEY, CLOUDINARY_CLOUD_NAME, CLOUDINARY_API_KEY, CLOUDINARY_API_SECRET, GOOGLE_DRIVE_CREDENTIALS_JSON, LINKPREVIEW_API_KEY, and VF_MEDIUM_API_KEY. The Google Drive variable contains the complete service-account JSON. Offline tests use a non-secret OpenAI placeholder; live integration tests need real credentials supplied outside the repository.

Tigris storage configuration (5.3)

The library reads IOptions<TigrisOptions> supplied by its host. Bind the Tigris configuration section before or after AddWikiImporter:

using VeeFriends.WikiImporter.Configuration;

services.Configure<TigrisOptions>(configuration.GetSection(TigrisOptions.SectionName));
services.AddWikiImporter(/* existing provider configuration */);

Hosts can also use services.Configure<TigrisOptions>(options => { ... }). The existing AddWikiImporter signature is unchanged. Both Tigris:AccessKeyId and Tigris:SecretAccessKey must be nonblank when the Tigris file store is resolved. Validation errors name missing settings without including credential values. The endpoint remains https://t3.storage.dev and the bucket remains wiki-store.

For Azure hosts, store the rotated credentials in Key Vault and create Azure App Configuration Key Vault references named Tigris:AccessKeyId and Tigris:SecretAccessKey. Load these through the host's existing App Configuration provider with Key Vault resolution and grant the host identity secret-read access. App Configuration stores reference URIs, not plaintext credentials. The library does not require process environment variables or an Azure SDK dependency.

The standalone CLI and integration-test host explicitly map TIGRIS_ACCESS_KEY_ID and TIGRIS_SECRET_ACCESS_KEY into these options. Offline tests use a temporary local file store when both variables are absent. Do not commit credential values or put real credentials in test fixtures.

Release this change as 5.3. Consumers using 5.1.* or 5.2.* must explicitly upgrade after preparing their host configuration; an unrelated restore must not silently adopt the new credential requirement. For veefriends.com, coordinate with PR #3053. Credentials are captured by the singleton client when it is created, so redeploy/restart the host after rotating them.