Guten Tag!

Many greetings from Munich, Germany. Modern data science is increasingly becoming a task of chaining together a bunch of LLMs and other cloud services. That’s why I will start to teach you the foundations of AWS so that you learn how to set up S3 buckets with R & the {paws} package.

Registration

The first thing that you need to do is to register with AWS. You can easily do so via their website. There, you’ll have to sign up using your billing information. Just make sure that you stay within the free tier and then your billing information will never be used.

Creating an S3 Client

First, we need a place to store files. That’s where S3 buckets come in (S3 = Simple Storage Service).

Now, because we want to use the AWS user interface as little as possible, we are actually going to use R to set things up. To do so, we need to install the {paws} package.

This is a huge package. So it will take a while to install. While it installs, you can look at its cute hex sticker.

Once the install process is finished, we need to define an S3 client. Sounds ominous but here’s how you can think of it:

All AWS services are their own little restaurant. And in order to dine at this restaurant, you need to have a client that you can take out to said restaurant. So if you want to dine at the S3 restaurant, you want to have an S3 client.

And with the {paws} package you can easily create the corresponding client. All you need to do is to call the s3() function. Typically, you’ll save this client in a variable using the service name.

Show all buckets

Now the s3 variable is actually an object that can call a bunch of different methods. One such method is the list_buckets() method.

Oh no! This doesn’t look right. It seems like we don’t have the necessary credentials.

Makes sense if you think about it. How the heck should R know that it needs to interact with our AWS account? We’ve never specified it. And that’s where the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY come in.

We have to first create them using the AWS UI in the browser. And once we have them, we can set them in our .Rprofile using the Sys.setenv() command. This will tell R about our AWS account. So let’s head over to AWS.

Creating an AWS user

You want to head to the IAM service. Here, IAM stands for “Identity and Access Management”. There, you can create a new user.

First, you’ll need to come up with a name.

Then, you’ll want to attach specific policies to that user. These are the things that the new user will have access to.

In the policies you can search for “S3” and then attach the “AmazonS3FullAccess” policy. This will allow the user to use S3 without any restrictions.

Of course, AWS allows to create granular strategies to avoid giving people full access. But I think for this demo, this is perfectly fine. Also, while we’re at it, let’s also give the user access to the Textract service which we will work with next week.

Once all of that is done, you can head over to the next step and create the user.

Getting user credentials

Congrats! You know have create an AWS user that can access S3 and Textract. Now, we need to get credentials for that user. These are the things we can stick into our .Rprofile.

Here’s what you need to do. First, open up the new user:

Then, you can head over to the credentials tab and click the “create access key” button.

There, you’ll need to select the option to let an application run outside of AWS. This makes sense since we want to let our R session on our computer use the credentials. Therefore, access from outside the AWS network must be possible with the credentials.

And once you click through the remaining steps (where you don’t need to do anything), you’ll have credentials:

You can now stick them into your .Rprofile file to set the environment variables. And while you’re at it, you should also define your default region. That should be something that’s close to you. For me that’s eu-central-1.

Retry the list buckets command

And now after restarting your R session and creating a new S3 client, you should be able to do what we tried earlier.

If you don’t have such a bucket yet, you can create one with the create_bucket() method.

Sweet! With that you have a bucket into which you can upload files. And to demonstrate that let’s take our invoice from last week. You know, this one here:

And to actually upload the file, we can use the put_object() method.

And to check that this worked, we could look into our bucket.

Nice! Look at that! Inside the Contents part of the output you can see the file. That means our upload was successful.

Perfect! With that we have paved the way to pass data to OCR services like AWS Textract. But that’s a story for next week.

As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts