Use Cases:
Offering a new product or capability:
When introducing a new service like speech recognition, a common design pattern is to start with a small amount of traffic and gradually increase it.
Automating or assisting existing tasks:
If a task previously performed by humans, like inspecting smartphones for defects, can be automated or assisted by a learning algorithm, a shadow mode deployment can be used. This involves running the algorithm parallel to human inspectors without using its output for decision-making initially.
Replacing an existing system:
When replacing a previous implementation of a machine learning system with a better one, a gradual ramp-up with monitoring and the option for rollback are essential.
Deployment Patterns
Shadow Mode Deployment:
In shadow mode deployment, a learning algorithm shadows human inspectors without making actual decisions. - This allows for data collection and comparison between the algorithm’s performance and human judgment. - Shadow mode deployment helps verify the algorithm’s accuracy before allowing it to make real decisions.
Canary Deployment:
Canary deployment involves gradually introducing a learning algorithm to make real decisions. - Initially, a small fraction (e.g., 5%) of traffic is routed to the algorithm to monitor its performance. - By starting with a small percentage of traffic, potential mistakes by the algorithm can be limited and closely monitored. This deployment pattern helps detect and address issues before they cause significant consequences.
Blue Green Deployment:
Blue green deployment is used when transitioning from an old version of software to a new one. - The old version (blue) and the new version with a learning algorithm (green) coexist. - Traffic is initially sent to the blue version, and then, when ready, it is switched to the green version. - This pattern allows for easy rollback to the previous version if needed.
Degrees of Automation:
Automation can range from no automation (human-only) to full automation (algorithm makes all decisions). Different degrees of automation include:
- No Automation
- Shadow mode
- AI assistance
- Partial automation involves the algorithm making decisions confidently and deferring to humans for uncertain cases.
- Full automation.
Human-in-the-loop deployments, such as AI assistance and partial automation, are often suitable for applications where the algorithm’s performance is not sufficient for full automation.