Finally, i was able to complete this! 
Using Below services from Amazon:
EC2- Compute instances: Used Amazon AMI, installed Apache, PHP and checked out drupal core files, converted to an image.
RDS- MariaDB dedicated instance
Autoscaling Group: Launch config & Auto Scaling Group, using the custom image created from EC2
ELB: Frontend load balancer for web servers
Code Deploy: To deploy the update code files to the web servers. Code deploy takes care of deploying the code to all new servers as it boot us on the autoscale group. (It starts from our internal gitlab server, once all files are committed, we will zip that solution and a gitlab runner will initiate a docker instance which then uses aws cli to push this zip file to S3, code deploy picks it up from there and deploy to the selected auto scale group)
Set threshold alarms: start with 1 server, when CPU reaches 80% and stays for a minute, boot up another instance. When CPU goes down for around 2 minutes, it will terminate 1 instance, to keep the minimal back to 1.
Testing: Using the Linux command stress.
Youtube Video
I will soon be writing a blog post on this, but just wanted to update all here 