I can't find my logs in AWS Lambda! (they might not be enabled...)

0

Full article

I setup an AWS Lambda job the other day, for cleaning up comments on SO, and then added a trigger to run it every morning. I checked it this morning and realized:

  1. The job failed for some reason.
  2. I had no idea what that reason was, because nothing was written to the logs.

According to the docs, all Console.WriteLine statements are logged:

To output logs from your function code, you can use methods on the Console class, or any logging library that writes to stdout or stderr.

But all I got on the logs page was an error message... an awful, unhelpful message:

There was an error loading Log Streams. Please try again by refreshing this page.

Telling users to refresh the page is like IT telling someone to turn their monitor off and back on - okay, it's something to do, but it's not going to fix any problems. Unless your problem is that your screen is dark.


Enable Logging

When you create a new Lambda job, logging is (inexplicably, to my thinking) not configured by default. I'm sure there's reasons for it, but considering this is a service where jobs run headless, it seems pretty important to be able to jump in and quickly see exactly why a job is failing. The metrics screens aren't enough.

So, if you've just setup a job and you're running into this same problem, check the execution role for your job, which "defines the permissions of your function". These permissions include the ability to write out to the logs (or a "log group" as they call it).

Look for this panel halfway down the screen for your function. Click the "View role" link.
You want to add a new policy, so click the big button that says "Attach policies".
Type "cloudwatchlogs" into the filter, select "CloudWatchLogsFullAccess", and attach it.
Verify the new policy shows up on the previous screen.
Run your job again, then click "View logs in CloudWatch".
You should see an entry, assuming your job wrote anything out.
Yay, logs. πŸŽ‰

My issue ended up being two separate problems:

  • I targeted .NET Core 2.1 when I created the Lambda function (because that's the only one available), but my C# project targeted .NET Core 3.0. Oops.
  • I also forgot to add a reference to Amazon.Lambda.Core, which is really easy to do since it's not used in the project nor required by any part of the project, but its absence will cause the job to fail when it runs on AWS. πŸ€¦β€β™‚οΈ

If that didn't do it for you, or you already had a comparable permission selected, there's some more helpful suggestions in this post by Dora Hodanic:

Amazon Connect and Lambda logs: Error loading Log Streams

Author

Grant Winney

Is there anything more satisfying than sharing knowledge? Of teaching someone and witnessing their "ah ha" moment? I usually write about tech, but no promises. I hope you find something interesting!



Comments