Heroku doesn't provide a native way of storing and retreiving old logs. Moreover, it doesn't provide any sort of monitoring or analyzing on the logs. This standalone app uses API Gateway and AWS Lambda to stream logs into CloudWatch in real time. After the logs are in CloudWatch, they can be further queried and analyzed by third party providers or even Cloudwatch Insights.
AWS API Gateway accepts the events from the Heroku Log Drain and parses it in a AWS Lambda. The logs can be further queried and analyzed by third party providers or even Cloudwatch Insights.
One advantage of this setup is that you can have your application emit logs with a minimum of blocking. At one point I had my application sending logs to Slack directly but this caused latency in the application any time I logged anything. By sending to Logplex on the other hand, I can process the application messages asynchronously without doing anything remotely fancy in my application. Another benefit is that you can handle your application, database, build, and deploy logs all the same unified fashion.
Prerequisites Setup AWS Credentials
Make sure you have AWS access key and secrete keys setup locally, following this video here
Download the code
git clone email@example.com:CodeRecipe-dev/Heroku-log-AWS-cloudwatch.git
Deploy to the cloud
cd Heroku-log-AWS-cloudwatch npm install serverless deploy --stage beta
Add endpoint as Log Drain on Heroku
heroku drains:add <api endpoint from sls deploy> -a <heroku app name>
After following the instructions above, any logs from heroku will also show up in CloudWatch for the Lambda function. Here's an example of what you will see in CloudWatch and its corresponding Heroku log:
Starting from the second line, exact same logs show up in CloudWatch as well