Creating a WinML Addon

This guide shows you how to create a C# native addon that uses Windows Machine Learning (WinML) in your Electron app. WinML allows you to run machine learning models (ONNX format) locally on Windows devices for tasks like image classification, object detection, and more.

Prerequisites

Before starting this guide, make sure you've:

• Completed the development environment setup
• Windows 11 or Windows 10 (version 1809 or later)

Step 1: Create a C# Native Addon

Let's create a native addon that will use WinML APIs. We'll use a C# template that leverages node-api-dotnet to bridge JavaScript and C#.

npx winapp node create-addon --template cs --name winMlAddon

This creates a winMlAddon/ folder with:

• addon.cs - Your C# code that will call WinML APIs
• winMlAddon.csproj - Project file with references to Windows SDK and Windows App SDK
• README.md - Documentation on how to use the addon

The command also adds a build-winMlAddon script to your package.json for building the addon, and a clean-winMlAddon script for cleaning build artifacts:

{
  "scripts": {
    "build-winMlAddon": "dotnet publish ./winMlAddon/winMlAddon.csproj -c Release",
    "clean-winMlAddon": "dotnet clean ./winMlAddon/winMlAddon.csproj"
  }
}

The template automatically includes references to both SDKs, so you can immediately start calling Windows APIs!

Let's verify everything is set up correctly by building the addon:

# Build the C# addon
npm run build-winMlAddon

Step 2: Download the SqueezeNet Model and Get Sample Code

We'll use the Classify Image sample from the AI Dev Gallery as our reference. This sample uses the SqueezeNet 1.1 model for image classification.

2.1. Download the Model

1. Install the AI Dev Gallery
2. Navigate to the Classify Image sample
3. Download the SqueezeNet 1.1 model (it supports CPU, GPU, and NPU)
4. Click Open Containing Folder to locate the .onnx file

5. Copy the squeezenet1.1.onnx file to a models/ folder in your project root

Step 3: Add Required NuGet Packages

Before adding the WinML code, we need to add additional NuGet packages required for image processing, ONNX Runtime, and GenAI support.

3.1. Update Directory.packages.props

Add the following package versions to the Directory.packages.props file in the root of your project (should have been created when you created the addon):

<Project>
  <PropertyGroup>
    <!-- Enable central package versioning -->
    <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
  </PropertyGroup>
  <ItemGroup>
    <PackageVersion Include="Microsoft.JavaScript.NodeApi" Version="0.9.17" />
    <PackageVersion Include="Microsoft.JavaScript.NodeApi.Generator" Version="0.9.17" />
    <!-- Add these packages for WinML -->
+   <PackageVersion Include="Microsoft.ML.OnnxRuntime.Extensions" Version="0.14.0" />
+   <PackageVersion Include="System.Drawing.Common" Version="9.0.9" />
+   <PackageVersion Include="Microsoft.Extensions.AI" Version="9.9.1" />
+   <PackageVersion Include="Microsoft.ML.OnnxRuntimeGenAI.Managed" Version="0.10.1" />
+   <PackageVersion Include="Microsoft.ML.OnnxRuntimeGenAI.WinML" Version="0.10.1" />
    
    <!-- These versions may be updated automatically during restore to match yaml -->
    <PackageVersion Include="Microsoft.WindowsAppSDK" Version="2.0.0-experimental3" />
    <PackageVersion Include="Microsoft.Windows.SDK.BuildTools" Version="10.0.26100.7175" />
  </ItemGroup>
</Project>

3.2. Update winMlAddon.csproj

Open winMlAddon/winMlAddon.csproj and add the package references to the <ItemGroup>:

<ItemGroup>
  <PackageReference Include="Microsoft.JavaScript.NodeApi" />
  <PackageReference Include="Microsoft.JavaScript.NodeApi.Generator" />
  <!-- Add these packages for WinML -->
+ <PackageReference Include="Microsoft.ML.OnnxRuntime.Extensions" />
+ <PackageReference Include="System.Drawing.Common" />
+ <PackageReference Include="Microsoft.Extensions.AI" />
+ <PackageReference Include="Microsoft.ML.OnnxRuntimeGenAI.Managed" />
+ <PackageReference Include="Microsoft.ML.OnnxRuntimeGenAI.WinML" />
  
  <PackageReference Include="Microsoft.Windows.SDK.BuildTools" />
  <PackageReference Include="Microsoft.WindowsAppSDK" />
</ItemGroup>

What these packages do:

• Microsoft.ML.OnnxRuntime.Extensions - Provides additional operators and utilities for ONNX Runtime
• System.Drawing.Common - Enables image loading and manipulation for preprocessing
• Microsoft.Extensions.AI - AI abstractions for .NET
• Microsoft.ML.OnnxRuntimeGenAI.Managed - Managed bindings for ONNX Runtime GenAI
• Microsoft.ML.OnnxRuntimeGenAI.WinML - WinML integration for ONNX Runtime GenAI

Step 4: Add the Sample Code

The AI Dev Gallery shows the complete implementation for image classification with SqueezeNet:

We've adapted this code for Electron and you can find the complete implementation in the electron-winml sample. The winMlAddon/ folder contains the modified code from the AI Dev Gallery.

Copy the entire winMlAddon/ folder from samples/electron-winml/winMlAddon/ to your project root, replacing the one created in Step 1. The sample includes multiple files beyond addon.cs (helper classes in Utils/, a chat client, etc.) that are required for the addon to build and run.

Key Implementation Details

Let's highlight the important parts of the implementation and key differences from the AI Dev Gallery code:

1. Project Root Path Requirement

Unlike the AI Dev Gallery code, our Electron addon requires the JavaScript code to pass the project root path. This is necessary because:

• The addon needs to locate the ONNX model file in the models/ folder
• Native dependencies (DLLs) need to be loaded from specific directories

[JSExport]
public static async Task<Addon> CreateAsync(string projectRoot)
{
    if (!Path.Exists(projectRoot))
    {
        throw new Exception("Project root is invalid.");
    }

    var addon = new Addon(projectRoot);
    addon.PreloadNativeDependencies();

    string modelPath = Path.Join(projectRoot, "models", @"squeezenet1.1-7.onnx");
    await addon.InitModel(modelPath, ExecutionProviderDevicePolicy.DEFAULT, null, false, null);

    return addon;
}

This automatically selects the best execution provider (CPU, GPU, or NPU) based on device capabilities.

2. Preloading Native Dependencies

The addon includes a PreloadNativeDependencies() method to load required DLLs. This approach works for both development and production scenarios without needing to copy DLLs to the project root:

private void PreloadNativeDependencies()
{
    // Loads required DLLs from the winMlAddon build output
    // This ensures dependencies are available regardless of the execution context
}

This is called during initialization before loading the model, ensuring all native libraries are available.

3. Configuring Electron Forge for Packaging

To ensure the addon works correctly in production builds, you need to configure your packager to:

1. Unpack native files - DLLs, ONNX models, and .node files must be accessible outside the ASAR archive
2. Exclude unnecessary files - Keep the package size small by excluding build artifacts and temporary files

For Electron Forge, update your forge.config.js:

// From samples/electron-winml/forge.config.js
module.exports = {
  packagerConfig: {
    asar: {
      // Unpack native files so they can be accessed by the addon
      unpack: "**/*.{dll,exe,node,onnx}"
    },
    ignore: [
      // Exclude .winapp folder (SDK packages and headers)
      /^\/.winapp\//,
      // Exclude MSIX packages
      "\\.msix$",
      // Exclude winMlAddon source files, but keep the dist folder
      /^\/winMlAddon\/(?!dist).+/
    ]
  },
  // ... rest of your config
};

What this does:

1. asar.unpack - Extracts DLLs, executables, .node binaries, and ONNX models to app.asar.unpacked/

   • This makes them accessible at runtime via file system paths
   • The JavaScript code adjusts paths automatically (see the app.asar → app.asar.unpacked replacement above)

2. ignore - Excludes from the final package:

   • .winapp/ - SDK packages and headers (not needed at runtime)
   • .msix files - Packaged outputs
   • winMlAddon/ source files - Keeps only the dist/ folder with compiled binaries

4. Image Classification

The ClassifyImage method processes an image and returns predictions:

[JSExport]
public async Task<Prediction[]> ClassifyImage(string imagePath)
{
    // Loads the image, preprocesses it, and runs inference
    // Returns top predictions with labels and confidence scores
}

The complete implementation handles:

• Image loading and preprocessing (resizing, normalization)
• Running the model inference
• Post-processing results to get top predictions with labels and confidence scores

Understanding the Code

The addon provides these main functions:

1. CreateAsync - Initializes the addon and loads the SqueezeNet model
2. ClassifyImage - Takes an image path and returns classification predictions

WinML automatically selects the best execution device (CPU, GPU, or NPU) based on availability.

Step 5: Build the C# Addon

Now build the addon:

npm run build-winMlAddon

This compiles your C# code using Native AOT (Ahead-of-Time compilation), which:

• Creates a .node binary (native addon format)
• Trims unused code for smaller bundle size
• Requires no .NET runtime on target machines
• Provides native performance

The compiled addon will be in winMlAddon/dist/winMlAddon.node.

Step 6: Test the Addon

Now let's test the addon works by calling it from the main process. Open src/main.js and follow these steps:

6.1. Load the Addon

Add the require statements at the top:

const winMlAddon = require('../winMlAddon/dist/winMlAddon.node');

6.2. Create a Test Function

Add this function to test image classification:

const testWinML = async () => {
  console.log('Testing WinML addon...');
  
  try {
    let projectRoot = path.join(__dirname, '..');
    // Adjust path for packaged apps
    if (projectRoot.includes('app.asar')) {
      projectRoot = projectRoot.replace('app.asar', 'app.asar.unpacked');
    }
    
    const addon = await winMlAddon.Addon.createAsync(projectRoot);
    console.log('Model loaded successfully!');
    
    // Classify a sample image
    const imagePath = path.join(projectRoot, 'test-images', 'sample.jpg');
    const predictions = await addon.classifyImage(imagePath);
    
    console.log('Top predictions:');
    predictions.slice(0, 5).forEach((pred, i) => {
      console.log(`${i + 1}. ${pred.label}: ${(pred.confidence * 100).toFixed(2)}%`);
    });
  } catch (error) {
    console.error('Error testing WinML:', error.message);
  }
};

Key points:

• The path adjustment (app.asar → app.asar.unpacked) ensures the code works in both development and packaged apps
• This accesses the unpacked native files configured in forge.config.js

6.3. Call the Test Function

Add this line at the end of the createWindow() function:

testWinML();

6.4. Prepare Test Images

To test image classification:

1. Create a test-images/ folder in your project root
2. Add a test image named sample.jpg (the code expects this exact filename)
3. The SqueezeNet model recognizes 1000 different ImageNet classes (animals, objects, scenes, etc.)

When you run the app, you'll see the classification results in the console!

Step 7: Update Debug Identity

To ensure the Windows App SDK is loaded and available for usage, we need to ensure we setup debug identity which will ensure the framework is loaded whenever our app runs. Likewise, whenever you modify Package.appxmanifest or change assets referenced in the manifest (like app icons), you need to update your app's debug identity. Run:

npx winapp node add-electron-debug-identity

This command:

1. Reads your Package.appxmanifest to get app details and capabilities
2. Registers electron.exe in your node_modules with a temporary identity
3. Enables you to test identity-required APIs without full MSIX packaging

Now run your app:

npm start

Check the console output - you should see the WinML test results!

Next Steps

Congratulations! You've successfully created a native addon that can run machine learning models with WinML! 🎉

Now you're ready to:

• Package Your App for Distribution - Create an MSIX package that you can distribute

Or explore other guides:

• Creating a Phi Silica Addon - Learn how to use the Phi Silica AI API
• Getting Started Overview - Return to the main guide

Customizing for Your Model

To fully integrate your ONNX model, you'll need to:

1. Understand your model's inputs - Images, tensors, sequences, etc.
2. Create proper input bindings - Convert your data to the format WinML expects
3. Process the outputs - Parse and interpret the model's predictions
4. Handle errors gracefully - Model loading and inference can fail

Additional Resources

• WinML Documentation - Official WinML documentation
• winapp CLI Documentation - Full CLI reference
• Sample Electron App - Complete working example
• AI Dev Gallery - Sample gallery of all AI APIs
• Windows App SDK Samples - Collection of Windows App SDK samples
• node-api-dotnet - C# ↔ JavaScript interop library

Troubleshooting

node -e "console.log(process.arch)"

# For x64 Node.js:
dotnet publish ./winMlAddon/winMlAddon.csproj -c Release -r win-x64

# For ARM64 Node.js:
dotnet publish ./winMlAddon/winMlAddon.csproj -c Release -r win-arm64

rm -rf node_modules package-lock.json
npm install

Get Help

• Found a bug? File an issue
• WinML questions? Check the WinML documentation

Happy machine learning! 🤖