AI Preset Detection

AI Preset Detection is a system developed by EyeQ to apply different presets to photos of differing subject matter, lighting conditions, or content. Instead of applying iAuto '21 to every image that you process - you can create a family of presets, each tuned for a specific scene. Scene Detection is also useful to simply learn more about the photos your customers are sending to your business - we call this "Image Intelligence". There are two main parts to AI Preset Detection:

1. Scene Detection using AI to detetmine the content of the photo
2. Automatic Preset Selection based on our carefully chosen defautls, or using presets you have tuned and edited in Workbench

There are over 20 Scenes currently defined in the SDK, and these are visible in the latest version of Workbench. You may edit the presets for each of the scenes, then "Export Presets..." to create a .preset file, which you can use in the SDK or Command Line Application.

EyeQ can create Custom Scenes designed for your business. Read more here

Note: Scene Detection is an optional component to our SDK. Contact your SDK account executive for pricing and availability.

Using AI Preset Detection

There are three main ways to use this AI Preset Detection component, from the most simple, to the most advanced:

1. The Command Line Application and Docker solutions includes support for AI Preset Detection

If your business opts-in for Scene Detection support, then this module will be enabled by default. See the documentation for the command line application for details.

2. Using AutoCorrect

This is the simplest way to enable Scene Detection using the Perfectly Clear SDK. The only additional step that is required is to simply create a PFCENGINE and then load the detection model. Once this is done, include the PFCENGINE when calling PFC_AutoCorrect.

Load the AI Engine:

    // Use AI-enabled PFCENGINE to do scene detection and apply a scene preset accordingly.
    // IMPORTANT: initialization of AI engine takes time, so better keep a SINGLE INSTANCE of the AI-enabled
    // engine per worker thread throughout your app session.
 
    PFCENGINE *pAiEngine = PFC_CreateEngine();
 
    // Load AI engine (assuming all dlls and models are in exe folder):
    std::string exePath = std::string(argv[0]);
    std::string binPath = exePath.substr(0, exePath.find_last_of('/')).c_str();
    int aistatus  = PFC_LoadAIEngine(pAiEngine, AI_SCENE_DETECTION | AI_CORRECTIONS, binPath.c_str());

Optionally, load custom Presets for the Scenes you will be detecting:

    // AI Scene Detection includes default presets for all scenes it defines.
    // These presets are immediately available after the call to PFC_LoadAIEngine().
    // One can return to these presets calling with a null pointer like
    //
    // <code>
    // bool retLoadPresets = PFC_LoadScenePresets(pAiEngine, nullptr);
    // <code>
    // You may modify these presets by editing them in Workbench, then exporting
    // a Scene Detection compatible .preset file then load those presets
    // with the following code.
    if (profilename == NULL)
    {
        if (bVerbose) {
            printf("Using default presets for detected scenes\n");
        }
    } else {
        bool retLoadPresets = PFC_LoadScenePresets(pAiEngine, profilename);
        if (0 != retLoadPresets) {
            printf("Cannot load scene presets file '%s' error '%i'\n", profilename, retLoadPresets);
            exit(1);
        } else {
            printf("Loaded scene presets file '%s'\n", profilename);
        }
    }

Call AutoCorrect to process image with the AI selected preset:

    // AutoCorrect with a valid AiEngine will ignore PFCPARAM and use AI determined presets instead
    PFCPARAM ignoredParam;
    int status = PFC_AutoCorrect(&im, NULL, ignoredParam, -1, NULL, false, NULL, pAiEngine);
 
    if (status == APPLY_SUCCESS && bVerbose)
    {
        printf("Image processed successfully.\n");
    }

Read the Scene that was detected:

    // Scenes are uniquely identified by an integer label.
    int lastSceneLabel = 0;
    int lastSceneConfidence = 0;
    PFC_GetLastSceneProperties(pAiEngine, &lastSceneLabel, &lastSceneConfidence);
    printf("AI assigned scene '%i' with confidence '%i'\n", lastSceneLabel, lastSceneConfidence);

3. Using separate Calc and Apply calls, and determining the detected Scene

This method allows more fine-grained control of the scene detection process, extracting which scene was detected, and optionally loading custom Scene Detection Presets.

Load the AI Engine, just like in the example above. Optionally, load custom Presets for the Scenes you will be detecting.

Run PFC_Calc to gather image stats and run AI Scene Detection Selection, saving the detected Scene in the PFCPROFILE for the image:

    // Analyze image and obtain image specific profile ad categorization.
    // For performance reasons CALC_ALL is kept to mean all classic corrections.
    // Scene Detection is it's own feature.
    PFCPROFILE pProfile = PFC_Calc(&im, NULL, pEngine, CALC_ALL | CALC_SCENE_DETECTION, -1, NULL, NULL, bVerbose? myStatus : NULL, PFC_REJECT_MONOLITH, pAiEngine);

Read the Scene that was detected from the PFCCPROFILE:

    // After detectionn the scene id and model version can be optained through PFC_GetScene/
    // The specific meaning depends on the model loaded.
    int version = 0;
    int detectedScene = PFC_GetScene(pProfile, &version);
    printf("AI version '%i' assigned scene '%i' to the image.\n", version, detectedScene);
 
    // Apply scene preset for detected scene by getting the params to apply from the AI-enabled
    PFCPARAM param;
 
    if (detectedScene >= 0) {
        PFCPARAM sceneParam;
        if (0 == PFC_ReadScenePreset(sceneParam, pAiEngine, detectedScene)) {
            param = sceneParam;
        }
    }
 
    // Its name can be obtained through PFC_GetSceneDescription. This should *not* be done in a tight loop.
    if(bVerbose) {
        PFCSCENEDESCRIPTION description;
        int i = 0;
        while(PFC_GetSceneDescription(pAiEngine, i, &description)) {
            if(description.profileUUID == detectedScene) {
                printf("Scene '%i' is named '%s'.\n", detectedScene, description.name);
            }
            ++i;
        }
    }

Then, process the image with PFC_Apply:

    // Process image with user or scene parameters.
    PFCAPPLYSTATUS status = PFC_Apply(&im, pEngine, pProfile, param, bVerbose? myStatus : NULL);
    // Optional display of status.
    if (status == APPLY_SUCCESS && bVerbose)
    {
        printf("Image processed successfully.\n");
    }
    else if (status > APPLY_SUCCESS && bVerbose)
    {
        // Check return code for Noise Removal
        printf("Noise removal status %d\n", NRRETCODE(status));
 
        // Check return code for Perfectly Clear Core correction
        printf("Perfectly Clear correction status %d\n", CORERETCODE(status));
 
        // Check return code for Face Beautification
        printf("Face beautification status %d\n", FBRETCODE(status));
 
        // Check return code for Red Eye Removal
        printf("Red eye removal status %d\n", RERETCODE(status));
    }
    else if (status < APPLY_SUCCESS )
    {
        printf("Image processing failed with return code:  %d\n", status);
    }

Copyright

Copyright (C) 2021 EyeQ Imaging Inc