Deploy an LLM API
Here, we’re using a modified version of the Python code found in this blog post
We are deploying a “question-and-answer” endpoint, using pytorch and the transformers library. As in the above blog example, we’re using the Wikipedia page on cars to test out our model/API.
- Deploy it now!
dan@node1:~/deploymenow_client$ dmn deploy sample_app_hello_world.py
.Your application is deployed!
...Now checking if your app is ready
..Everything is ready!
{"url": "https://molecular-onion-176209579.deploymenow.com", "message": "your API is ready on our end!...but if you see a 404 in your browser, or an SSL-error, give it a few moments! Your browser may need a moment to catch up :) ..."}- Let’s test it out!
curl -s --header "Content-Type: application/json" --request POST --data '{"question": "What was one of the first cars accessible to the masses?","context": "Cars came into global use during the 20th century, and developed economies depend on them. The year 1886 is regarded as the birth year of the modern car when German inventor Karl Benz patented his Benz Patent-Motorwagen. Cars became widely available in the early 20th century. One of the first cars accessible to the masses was the 1908 Model T, an American car manufactured by the Ford Motor Company. Cars were rapidly adopted in the US, where they replaced animal-drawn carriages and carts, but took much longer to be accepted in Western Europe and other parts of the world."}' https://molecular-onion-176209579.deploymenow.com|jq
{
"Question": "What was one of the first cars accessible to the masses?",
"Answer": "1908 model t"
}Wow, that was easy!