Struggling to Deploy a Scalable ML Model with AWS ECS and Terraform for Mobile Optimization
I'm optimizing some code but I'm experimenting with I'm dealing with Hey everyone, I'm running into an issue that's driving me crazy..... Quick question that's been bugging me - I'm working on a personal project and Trying to implement a scalable deployment strategy for my machine learning model using AWS ECS with Terraform. The objective is to ensure that the model is optimized for mobile access, but I've run into some complications regarding resource provisioning. My current setup involves using the following Terraform configuration: ```hcl provider "aws" { region = "us-west-2" } resource "aws_ecs_cluster" "ml_cluster" { name = "ml-cluster" } resource "aws_ecs_task_definition" "ml_task" { family = "ml-task" requires_compatibilities = ["FARGATE"] network_mode = "awsvpc" container_definitions = jsonencode([ { name = "ml_model" image = "myrepo/ml_model:latest" cpu = 256 memory = 512 essential = true portMappings = [ { containerPort = 80 hostPort = 80 protocol = "tcp" } ] } ]) } ``` The ECS service is designed to automatically scale based on incoming requests, but I noticed that, during peak times, it sometimes fails to launch enough tasks to handle the load. I’ve tried adjusting the `desired_count` parameter and using CloudWatch alarms to trigger scaling policies, but I’m unsure if my monitoring setup is correct. Here's the auto-scaling policy I implemented: ```hcl resource "aws_appautoscaling_policy" "service_scale_out" { name = "scale_out" policy_type = "StepScaling" resource_id = "service/${aws_ecs_cluster.ml_cluster.name}/${aws_ecs_service.ml_service.name}" scalable_dimension = "ecs:service:DesiredCount" step_scaling_policy_configuration { adjustment_type = "ChangeInCapacity" step_adjustments { scaling_adjustment = 1 metric_interval_lower_bound = 0 } cooldown = 300 } } ``` Despite this, I still encounter issues with task placement and resource constraints, particularly with the network mode. The model needs quick responses from multiple mobile users, but I end up with throttled requests under heavy load, leading to performance degradation. I’m also worried about the cost implications of underutilizing resources or over-provisioning. Has anyone worked through a similar setup and managed to optimize it successfully for mobile-heavy traffic? Any insights or best practices regarding ECS configurations, scaling policies, or even the task definition would be greatly appreciated! I'm working on a CLI tool that needs to handle this. How would you solve this? I'm working on a service that needs to handle this. Any ideas what could be causing this? My team is using Hcl for this web app. Any feedback is welcome! I'm on CentOS using the latest version of Hcl. Cheers for any assistance!