How We Built AlgoExpert's Remote Code Execution Engine

How We Built AlgoExpert's Remote Code Execution Engine

SUBTITLE'S INFO:

Language: English

Type: Robot

Number of phrases: 323

Number of words: 2397

Number of symbols: 10271

DOWNLOAD SUBTITLES:

DOWNLOAD AUDIO AND VIDEO:

SUBTITLES:

Subtitles generated by robot
00:00
where is this Florida it's you guys ya know what don't know what's up everybody Antoine here with Clement know because you know what's up everybody how's it going what's up everybody how's it going Antoine here and I'm with Clement my co-founder algo expert we've got a very exciting video for you today I'm here with my co-founder on algo expert hands on and we are gonna be talking about a very cool feature that we've gone now though expert view remote code execution engine we'll get into what that is in a second if you don't already know and
00:32
we're gonna be talking about how we built it but before we do antoine what's the obligatory a go expert plugin so go to algo expert dot io / Klem and you can use Clement as a promo code on the platform floor this guy it's the worst pitch ever take the promo code Plan C lem for a discount on the platform thanks that's way better that was like you know that like people are gonna be more inclined to go anyway so on algo expert users can write code and run it
01:03
against pre-made test cases they can also run it against their own test cases how does that work okay well so the user writes code in the UI right and when they press run code UI takes that and sends it over to the server as an HTTP request and the server sees that and I'm gonna simplify it for now but essentially calls out to Python or nodejs or the Java compiler and runs that code captures the output and then
01:33
sends it back to to the UI yeah that's basically what we call the remote code execution so exactly now the expert we call it the RCE now this is like a critical part of algo expert it's a critical part of the platform it sounds simple enough like the way you describe it is basically an HTTP request that has an input and then it spits something back out right is it that simple no well so it started out really really simple which we basically did that for at least
02:04
the proof-of-concept work that we did a while ago almost three years ago now I guess yeah but since then we've had to handle a lot of edge cases and it's called for a lot of improvements over time when you only consider the happy path right of someone runs code and the code compiles and the code runs and code passes test cases it's it's really easy to do right but what's tricky is getting the more the
02:34
edge cases references which are not even that Ed's know they happen all the time but you have a lot of users we have tens of thousands of user run code on our website every single month right or every week now what's the I think on a daily basis these days we have Lily 8,000 run codes yeah yeah no service code run code 8,000 times every single day and we have a lot of different kinds of questions on the website so we hid those that cases very very often right those edge cases or something things like running out of memory or having an
03:06
infinite loop right so the code times out you need to make sure that you clean up the process is properly on the back end or you need to make sure that the process doesn't use as much memory as the machine has is that otherwise we can't serve traffic right right so we have to be able to isolate one users code from another users cook so that's the key thing I think that people need to wrap their head around if they're trying to understand how this RC works is that what you just said isolation you can't have one users run code processor
03:36
or like same effect another user yeah exactly and so what is like how do we do that basically so actually since the get-go the main the core technologies that we use to do that it's called docker it's an open source technology it's very very popular among infrastructure engineers very very easy to use you someone who doesn't know much about a lot of operating system level things or kernel level things right and it gives you most of the tools that you need to be able to isolate a process from one other in
04:08
terms of files in terms of CPU this is using a docker container yes you can you basically containerize someone's run code event or environment in a single invite isolated environment and that allows us to do that now let's assume that someone knows how to use docker and ow Italy you know again it's almost like once you once you get past that hump sounds simple enough but again like what are the challenges that come with this so so the second iteration of the remote code
04:40
execution just spun up a new container right we'll created a new container for every single run code so again just to clarify you you make an HTTP request you spin up a new docker container with the code where you execute it in let's say node or Python or Java and that's that's what it is yeah and then when the code exit the container dies and gets cleaned up and everything happens and it's great except that the latency for users wasn't where we wanted it to be right because creating a new container takes a lot of time and there's a quick side note here
05:11
I want to point out something you all you Java users on algo expert are gonna appreciate this so one of the other challenges that you sort of hinted at is that we have to support multiple languages right now we really have six languages we have a seventh one coming soon comment down below if you know which one it is some languages like Java Script were pretty fast but Java was actually a very slow linguist at first it was very painful to optimize Java but we haven't sorted down now yeah we have it used to be remember the first time we got Java working or you got Java working
05:43
you came to me you're like clinic Java runs like it runs in the containers and everything and I was like how how long does it take to run a Java you know like to run D get and fib question in Java really 30 seconds yeah so obviously like I had got networking just the functionality but we were and we are still on really lean infrastructure we're not spending that much money so it is something that we've done intentionally right keep our costs down yeah and so it means we can't we can't
06:13
just use that as much CPU as we want to to make trial run really quickly like on your laptop's right you have like 8 CPUs and we don't want allocate CPUs just to run you know every sink every second code so so then how do we how do we optimize that because now Java actually runs like what's the 90th percentile of we're around 1.5 seconds for Java yeah where does like other languages are like javascript is like less than half a second which is great javascript is like our baby it's the one that were Union languages it was super it was super easy to containerize super
06:45
easy to run everything's been really great for working in JavaScript on a go expert you're gonna have really fast run code yeah the fastest basically on our plus the fastest on the platform but so Java how did we how did we optimize like were there any optimizations that we did yeah so there are a lot of language specific optimizations that already had to do so for Java for this video who know Java it's gonna sound familiar you you if you want to run the Java compiler you have to spin up or create a virtual
07:16
machine inside your container which is very very heavy-duty and so that takes a really long time and then to run the Java code you have to run that same virtual machine again and so I did a lot of optimizations around that to make sure that we only created as many as we needed it there as few as possible and so right now we only create I think one for every run code so that was one of the optimizations before Java and then but then for other languages it mostly had to do with the testing frameworks so Python I remember was one of them where we started out by using what PI test
07:48
yeah so we because I am a fan of things that already exist I don't want to rewrite things you have to so we started using PI test and it took three four seconds to run code just because PI test is a pretty heavy-duty framework and so pretty quickly I realized that we couldn't really do that because user experience is really important on yahoo expert we want the platformer feel really responsive yeah to be clear again it's having code running three or four seconds it's not necessarily the end of the world yeah but when we wanted to
08:20
build the best platform possible so we wanted everything we wanted the core product which is the coding workspace to be as fast as possible yeah so our our ideal was always under two seconds yeah good yeah but so we switched the testing framework for Python yeah it's the standard unit testing library for Python that we're using okay and that one's faster and that one's much less yes so another thing that might be interesting to some of you is that if you go now go expert and you run code you have two different types of output you have what we call the raw output this is what you give us right
08:52
the what the back-end gives us you run your code and you see like oh you have an error or all your tests are passing but then we also have the default output which we call the custom output where just says you know test case 1 2 3 4 5 grade they're all passing or their some of them are failing how do we do that well so at the very beginning it was very hard for us or I guess for me to conceptualize a world in which other languages can kind of coexist and have a similar fuel at least from an API point of view because because that is custom
09:22
output yes something that we don't get for free like the raw output yes so you have to somehow unify all languages and since we were just starting out on algo expert we died we didn't really have a good solution for that so Graham you did was actually take the raw output and parse it on the front on the front end and just uh and figure out which test cases passed and failed and reorder them in the proper way if some of you who've ever wondered like will ever be writing algorithms in production code well we actually had to write sort of string matching algorithms where we had one
09:54
string matching algorithms for each of the languages one for JavaScript one for Python because because all of the raw outputs from these various languages look too different because they were using their own testing frameworks and so I think I wrote like half of them that you wrote the other half though and we were just you know parsing the outputs and or even though the string outputs to figure out which test cases passed and failed and weave into all sorts of little issues with that so recently we decided to do this what we deemed to be the correct way and have
10:25
the back end right but so the way that this had like the would the right way of the severan isn't just for the back end to do the parsing of the arab put because it's it's pretty brittle and you can easily like create you can easily break it if the user outputs a certain string it can just trick kind of the backend yeah whatever is parsing it into thinking that a test pass really failed but so the right way of doing it which is the way that it took a while but we have now is we instrumented the testing frameworks in every single language to
10:56
make sure that the backend could very reliably in a parse essentially what the true output of your test cases were then relay that to the front end in like a very digestible format and so now this basically makes the front-end do a lot less work which is it's better it's better for the Frenette front-end not to have to parse the output but we just get sort of a list of results is something like that that tells us hey these are the you have you know 15 test cases the first five failed in the last 10 past as
11:27
a matter of fact we recently like a couple days ago deployed a new version of the of the user interface on algo expert which now relies on this back-end you know effort rather than the front-end parsing yeah but so that's it that is the algo expert remote code execution engine in just a little over ten minutes hopefully you found it informative Goku you were able to get a sense of how this was built it's been and continues to be a really interesting feature on algo expert that has both back-end and front-end ramifications and
11:58
infrastructure ramifications let us know what you think about it in the comments below let us know if you have any questions smash the like button if not for me for Antoine he loves the light thank you guys I really do and we'll see you in the next video

DOWNLOAD SUBTITLES: