Guten Tag! 👋 

Many Greetings from Munich, Germany. Welcome to another episode of me rambling about magic tricks. Okay, maybe the magic trick parts is really only in here because I thought I was clever last week and combined the {mirai} package with the word “Magic”.

But I’m not going to back down on this. Mirai is a magical package…until it is not.

Last week, I gave you the most basic examples of Mirai to highlight what it can do in principle. But as real world teaches you: It’s all fun & giggles until you actually try to get something done.

So today, let’s save ourselves from some dread to keep the Mirai magic alive. Let’s do it!

A very simple pitfall

Ok, so let’s create some Mirai tension. Let’s crash some async processes. And I don’t have to do much to do so. Check out this code that summarizes the price of diamonds from the diamonds data set:

Looks pretty easy and innocent, right? Well, let’s try this with Mirai:

Error handling

That’s not what we were looking for, were we? Looks like for some reason, the diamonds data set is not available. But it gets even worse:

This is not an actual error and your R process does not get stopped when Mirai returns a “miraiError”. You can see that by the fact that subsequent calculations still run.

See how the subsequent calculations run (and fail)? Here. this means that m[] did not actually return the number we were looking for. Consequently, multiplying that non-number by 2 bamboozles R into an error state.

Error check

One way to check that the Mirai calculation actually worked is to ask it:

That way, you can raise the error yourself:

Now, if you’re running many calculations with mirai_map() and don’t want to manually check for errors, there’s a nifty shortcut that you can use next to .flat (which we learned about last week.)

And in case you’re wondering: Only one iteration would crash and this is how it would look if we don’t stop at the first sign of trouble:

Be explicit

Cool beans! We detect errors like a pro now. That’s an important skill, you know. But I’m guessing you rather want to avoid errors entirely.

Well, I hate to break it to you. You can’t. Errors are part of the game. Get used to it.

Don’t worry, though. You can still build a career out of producing all sorts of errors. I did too 🤪 

But we should probably try to avoid producing errors if we can help it. So let’s avoid this particular error type. And it’s not that hard. All you have to do is to explicitly tell Mirai where diamonds comes from.

Ahh much better. Everything runs smoothly after telling Mirai that diamonds can be fetched out of the ggplot2 package (using the full name via the :: notation.)

Transfer information to workers

The reason why we had to be explicit here is because each Mirai worker comes equipped with a blank R session. This means no packages loaded! And that’s why you have to tell Mirai into which package it needs to reach to get the data.

But what if you want to use something that’s not accessible via a package? For example, this could be an intermediate thing you’ve calculated.

Same error as before. And it’s the same reason: The Mirai worker has no idea what calculations you ran before. After all, it starts fresh.

This time, though, we can’t point to a package and say “Mirai, pretty please, stop weaponizing your incompetence and just look at that package. The machine you’re running on knows that one.” 

That’s a shame, I know. Instead, we have to pass previously calculated variables along to the Mirai worker. We can do so by listing those after the expression (the part in the curly brackets):

Nice! Now our worker knows about df_pricey_diamonds. We could even change the naming along the way:

And we could even pass along everything in the current environment (but that’s discouraged):

Set up your worker

Finally, there’s an even more elaborate way to set up your Mirai workers. You can run the everywhere() function to run some code on all workers. For example, this can be used to make packages available.

See how the code inside the mirai() call used lots of stuff from the Tidyverse and it didn’t cause a problem? That’s because the workers were set up to run library(tidyverse) at the beginning of their lifetime.

Just to state the obvious: This is pretty useful. Especially when you have some tricky objects like database connections. But let me tell you about that particular magic next time.

Hope you enjoyed this week’s newsletter. As always, feel free to hit the reply button to share your thoughts.

Happy Coding and see you next week,
Albert

Whenever you’re ready, there are three I can help you:

1. Automate Your Data Reports: This course helps data analysts eliminate manual copy-paste reporting by automating PDF reports end-to-end, saving hours every cycle and preventing costly mistakes. (Using the lovely Typst language 😍)

2. Generate Insights in Minutes, not Hours: This comprehensive course teaches you to handle data faster, smarter, and more efficiently.

3. Bespoke Data Science Solutions: I’ve helped clients build their own data science solutions. Whether building custom web apps, PDF reports, AI automations or teaching workshops, I’ve got you covered. You can reach out to me via this form (or simply hit reply to this email)