Summer Internship – QA LLM fine-tune/prompt-engineering on clinical text notes


Email Alerts

Subscribe to be the first to know about new Prompt Engineering Jobs

    We respect your privacy. Unsubscribe at any time.
    Promptly Health

    Promptly Health

    - Internship

    🌎 Remote

    Posted on: 11 June, 2024

    Summer Internship – QA LLM fine-tune/prompt-engineering on clinical text notes

    What’s Promptly in a nutshell

    We are a Data Analytics company, collecting, integrating, and analyzing data on the outcomes of care.

    We answer the question – is this patient treatment the best it could possibly be?

    What’s our purpose

    We exist to empower every patient and every health organization on the planet with evidence on the outcomes of care!

    Why do we get up in the morning? Well, most healthcare professionals have chosen to work in healthcare driven by their desire to make a difference in patients’ lives. And so have we! We have chosen to follow this calling by addressing the biggest problem in healthcare: the lack of real-world evidence on the outcomes of care.

    For us, society denying patients better care due to lack of access to data is unethical, in a world where technology has improved so many aspects of our world. Making the right evidence available to healthcare organizations to help prevent one lost life, one care complication, one failed treatment is the moral obligation that big tech companies have – it’s our Hippocratic Oath.

    At Promptly, everything we do is driven by our core purpose: to promote better healthcare at lower costs for patients every day, by making health outcomes available.

    The Momentum

    We are scaling fast and (certainly not) breaking things, backed by renowned VC investors. In the next two years, we will enter new markets in Europe and Latin America, reaching out to new patients with multiple healthcare needs, whom we will help live their lives to their fullest potential. Exciting stuff!

    Summer Internship Details

    • Duration of 8 weeks, starting in end of june/beginning of july.
    • Work in a project that will help Promptly achieve long term objectives.
    • Full-time internship. You manage your own time.
    • If you have a holiday in the middle of the internship that’s ok.
    • Remote, but we have our offices open for you.
    • You will have a mentor who will be responsible for guidance and supervision.

    Project Introduction

    There is a lot of unstructured clinical data lost on thousands of free text notes. At Promptly we have CROs which already give us the correct answers to the questions, and access to the free text notes that also have the answer to those questions. The free texts are the contexts, which contain the various answers to the questions in CROs. Having this, we have the data needed to either prompt-engineer an LLM or even fine-tune an LLM of our choice for QA tasks like these.

    Project inserted in the Data Team.


    The objectives of this project are:

    • Understand our challenges when it comes to CROs answering from free text notes
    • Explore available technologies to tackle this challenge

    Intern Tasks

    The intern will be responsible for the following tasks:

    • Exploratory data analysis on the CROs and free text datasets.
    • Prompt-engineer a commercial LLM or open-source LLM (like Llama3)
    • Fine-tune a smaller LLM for QA
    • Test the approaches (AUC, Recall, Precision)
    • Weight in the best performant option (architecture challenges, advantages/disadvantages, costs, future works)

    Expected outcomes

    At the end of the internship, it is expected to achieve the following outcomes:

    • Report detailing rival approaches, advantages and disadvantages and achieved metrics on our own data.

    Technology Stack:

    • Python
    • Langchain
    • Pytorch
    • SentenceTransformers


    • Summer Internship
    • Remote-first, but you are welcome to go to our offices.

    What we offer:

    • A salary of 300€/month + meal allowance
    • The opportunity to work at a real-world startup that is already changing patients’ lives.
    • Work with and learn from an experienced team.

    What is the recruiting process like:

    • Internal triage of CVs.
    • Schedule an interview with our team to get to know you and to give you an overview of Promptly.
    • Final Selection of students.
    • Finally, you will get your offer and hopefully you’ll accept it!

    Intern Profile

    • Data scientist
    • Data scientist – ML specialist

    Our Culture & Values


    If there is 1 rule at Promptly, this is the one – before making any judgement, put yourself in the shoes of another.

    Empathy is the courage to choose to actively understand what the experience of another person could have felt like, without necessarily have experienced it ourselves.

    Empathy is not a one-day journey, it is a mindset that must be continuously developed – it is an everyday choice to fight for kindness and, by doing so, to help change people, organizations, and society for the better.

    If you embrace this mindset, you are a true leader @ Promptly, you are a true leader in our society.


    At Promptly, we don’t foster fixed hierarchies or top-down decision making. If you want to hit the ground running in our team, do not wait for someone to “tell you what to do”.

    Acting like an owner means that you should align with our core purpose and find the best way to deliver on it, every day.

    Starting a new project, analyzing new data, reaching out to a new client, there is no area of the company you cannot have an impact in, because there is no such thing as “someone else’s problem”.


    Working to improve healthcare is a great responsibility and what inspires us to not settle for less.

    You have the responsibility to be a leader within our company. A leader is not a title. A leader is someone who delivers on our core purpose and inspires others to follow our cultural values. It is a “great power, and a great responsibility”.

    Leadership comes with the awareness of one’s own strengths and weaknesses and the responsibility to use these to deliver on what you promised. It comes with the responsibility to understand the context, ask for help when needed (teamwork), and be accountable for the quality of what we produce (excellence).


    Our team is our greatest asset, and we are fortunate to attract great people. We go the extra mile to find the best person for every role – making our team diverse in backgrounds, personal experiences, skills and knowledge.

    If you are not contributing – or worse, actively destroying – teamwork, you are derailing us from our core purpose and ultimately hurting patient care.


    At Promptly, we’re recognized by society as thought leaders, contributing to research and evidence generation through technology and data science.

    Data Privacy

    Your personal data will be processed for the purposes of managing Promptly’s recruitment related activities which includes setting up and conducting interviews and tests for applicants, assess and review such candidates and similar activities needed in the recruitment and hiring process. Your personal data will be shared with Proef, a provider engaged by Promptly to help us manage our recruitment and hiring process. For more information about how Promptly processes personal data and information about your rights etc., please see Promptly’s/ Proef’s Privacy Policy (link to: Policy for management of application (

    By submitting your application, you consent to Promptly/ Proef processing your personal data for recruitment purposes and to store such personal data for up to 12 months, unless we receive a request of erasure of the data from you to, in case Promptly would like to contact you regarding future job openings.

    Share the job:

    Related Jobs

    Prompt Engineer
    Prompt Engineer

    Full Time -

    AI Engineer
    AI Engineer

    Full Time -