AWS ELB and Secure WebSockets with Play Framework

UPDATE 2016/08/10: AWS improved their ELB and they now support websocket and http/2 protocols. More on this link https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/.

AWS ELB does not support WSS protocol on its HTTPS endpoints. If you are using it for load balancing this becomes a blocker for scaling your service. Hopefully there is a way to overcome this limitation.

Switching ELB protocols to TCP/SSL will not be enough as the server will not receive X-Forwarded-For header anymore.

To solve this you will need to

Add ProxyProtocol policy to ELB so it starts using proxy_protocolEnable proxy_protocol support on nginx (Play unfortunatelly cannot be configured to understand proxy protocol at this moment :( )Proxy protocol adds additional header to requests to pass server client’s ip which can be used if there is a load balancer between your server and clients.

How it looks?

PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP + single space + PROXY_IP + single space + CLIENT_PORT + single space + PROXY_PORT + "\r\n"

Configuring ELB

Adding policy to ELB is for now only available through aws-cli. You can download it from here.

1. Add Policy

aws elb create-load-balancer-policy \
   --load-balancer-name <AWS_ELB_NAME> \
   --policy-name My-proxy-protocol \
   --policy-type-name ProxyProtocolPolicyType \\
   --policy-attributes AttributeName=ProxyProtocol,AttributeValue=True

2. Configure ELB

3. Configure nginx on your instance

Enable proxy_protocol support on nginx on your instance (and use it as a template for any other instance in your scaling pool).

Important notes:

• We use Nginx here to pass requests to Play running on port 9000 on the same instance.
• Port 80 is kept here to redirect requests to https and it cannot use proxy protocol as ELB will not enable proxy protocol on HTTP/HTTPS listeners (which is a must on port 80)

Here is a sample nginx configuration for port 80

# Redirect everything to https
server {
       listen 80 default_server;
       listen [::]:80 default_server;

       server_name _;

       return 301 https://$host$request_uri;
}

4. Attach policy to ELB

Attach policy to ELB so it adds proxy protocol to all requests that are passed to port 81 on the instance

aws elb set-load-balancer-policies-for-backend-server \
   --load-balancer-name <ELB_NAME> \
   --instance-port 81 \
   --policy-names My-proxy-protocol

5. Open port 81 for EC2 security group :)

Just a reminder :)

6. Test it out

Test your setup by using 'wss://' in your requests. Websocket.org gives you an easy way to test websockets.

‍