Define success in plain language
Write down the event, where it should be detected and what the operator should do next. “Better security” is too broad to test. “Flag a person crossing this restricted boundary after closing, with a source image for review” gives the pilot a concrete purpose. Agree the rule with the people who will use it.
Use footage that reflects the site
Select representative camera views and operating periods. Include ordinary activity as well as the events of interest, and document conditions such as poor light or occlusion. Avoid choosing only easy examples after seeing the results. Keep any evaluation footage within the access and retention arrangements agreed for the project.
Record both kinds of mistake
Review alerts that do not match the intended event and events the system misses. Keep uncertain cases visible instead of forcing them into a success category. A model confidence score describes an output; it is not a measured accuracy rate for the deployment. Where possible, have a human reviewer create the comparison record before inspecting the model results.
Test the whole response path
Measure the experience from the visible event to notification, review and acknowledgement. Check whether the event reaches the right person and whether enough context is available to act. Include a test of camera loss, network interruption or an unavailable notification channel, with an agreed fallback procedure.
Make a documented rollout decision
Review results with the operational owner. Record what passed, what needs camera or rule changes, and what remains out of scope. Include training, maintenance, permissions and review intervals in the deployment plan. Reassess after material changes to camera position, lighting or site activity.
This guide helps structure a project conversation. Confirm hardware, configuration and commercial scope for your own site.