Automatically breaking down a user stories into sub-tasks? Creating test cases? Checking requirements quality? All of this can be done quite well by ChatGPT and Large Language Models (LLMs) in general. But how can we integrate an LLM so that we have this functionality directly integrated in Polarion? This post and the upcoming ones will guide you step by step on integrating a LLM for your use case. We’ll use as an example a form extension that allows you to break down user stories into individual tasks using AI. By following this blog series you’ll not only enable yourself to automate tedious tasks in Polarion with AI, but also become a technical expert on LLM integration in general. So stay tuned and don’t miss out on the next posts!
Precondition: A credit card and a few dollars to spend on AI services. (1 dollar is more than enough)
Table of Contents:
- Overview
- Creating a Velocity Form Extension
- Enabling REST API
- Set up OpenAI Account
- Getting the OpenAI API Key
- Integrating GPT-4o into Form Extension
- Complete Code
- Improving Outputs
- What to Consider for Production
- Summary
Overview
How does an LLM integration work? Think of using ChatGPT. But instead of creating the prompts yourself, you create the prompt automatically using some logic. The prompt that get’s created when clicking the „Generate Sub-Tasks“ button will build a prompt that could look like this one:
The prompt that will be sent, will consist of two main parts:
- System Configuration: Explains what the system shall do. This is where the main „prompt engineering“ takes place. This configuration is usually invisible to the user and is also seperated from the main prompt. In this case, that the system shall break-down the user story into sub-tasks.
- User Prompt: Contains the user story description. In our case this will be automatically the currently opened user story description.
And instead of getting a simple text message back, you’ll get a „machine readable“ answer in JSON or XML back, which can then be parsed and rendered the way you need it:
You may or may not ask „is it really that simple?“ and for that question, the answer is: mostly, yes! Of course there are multiple „levels“ to it. This shall become a series in which I’ll explain more and more details to cover increasingly complex use cases. In this post we’ll start with level 1. A very simple prototype of a simple use case.
Creating a Velocity Form Extension
Polarion offers great customizability by providing APIs (REST, Rendering & OpenAPI) and the ability to „inject“ HTML, JS & CSS into the front-end with simple Velocity scripts. This allows to integrate new functionalities like an „AI Task Breakdown“ into the work item view:
How to create such a form extension, how to deploy it and how to use REST API can be read in this previous post in detail: Polarion REST API: A Form Extension Example
In this post I won’t go too deep into how to create a form extension and how to develop a velocity form extension. Nevertheless, I would like to make a step by step guide to make it easier to follow. Therefore, I’m using the „E-Library“ (Agile) demo project configuration that comes with Polarion. This means that you can add the code snippets shown below into your E-Library project and it shall work. (If you add your own LLM correctly as described in this post)
The following code snippet is used as a starting point. It allows to show linked tasks in a table:
#######################
## Main
#######################
## Configuration and Context Variables
#set($taskWorkItemType = "task")
#set($workItemId = $object.getId())
#set($workItemTitle = $object.getTitle())
#set($workItemDescription = $object.getDescription().getContent().replaceAll("text/html: ",""))
#set($projectId = $object.getProjectId())
## Get current Work Item as Rendering API Object
#set($renderingWorkItem = $transaction.workItems().getBy().oldApiObject($object))
#set($linkedWorkItemsQuery = "SQL:(select WORKITEM.C_URI from WORKITEM inner join PROJECT on PROJECT.C_URI = WORKITEM.FK_URI_PROJECT inner join STRUCT_WORKITEM_LINKEDWORKITEMS as SWL on WORKITEM.C_URI = SWL.FK_URI_P_WORKITEM inner join WORKITEM as WORKITEM2 on WORKITEM2.C_URI = SWL.FK_WORKITEM where PROJECT.C_ID = '$projectId' AND WORKITEM.C_TYPE = '$taskWorkItemType' AND WORKITEM2.C_ID = '$workItemId')")
#set($linkedTasks = $trackerService.queryWorkItems($linkedWorkItemsQuery, "created"))
<table id="subTaskTable" class="polarion-rpw-table-content">
<tr class="polarion-rpw-table-header-row">
<th style="min-width: 50px;">ID</th>
<th>Title</th>
<th>Description</th>
<th style="min-width: 70px;">Status</th>
<th>Action</th>
</tr>
#foreach($linkedWorkItem in $linkedTasks)
<tr class="polarion-rpw-table-content-row">
<td style="min-width: 50px;">$!linkedWorkItem.getId()</td>
<td>$!linkedWorkItem.getTitle()</td>
<td>$transaction.workItems().getBy().oldApiObject($linkedWorkItem).fields().description().render().htmlFor().forFrame()</td>
<td style="min-width: 70px;">$!linkedWorkItem.getStatus().getName()</td>
<td>Already Existing</td>
</tr>
#end
</table>
<br>
This velocity script is stored under „[POLARION_INSTALL]/scripts/workItemForm_simpleAiTaskBreakdown.vm“. Then the form extension is added to the form configuration of the user story:
<extension collapse="true" expand="false" id="velocity_form" label="AI Task Breakdown" script="workItemForm_simpleAiTaskBreakdown.vm"/>Enabling REST API
We want to make the form extension interactive by allowing the user to create proposed tasks as linked Work Items. To achieve this, we will use the REST API, which must have the „X-REST Token“ enabled in the Polarion properties first. Add the following properties to the „[POLARION_INSTALL]/polarion/configuration/polarion.properties“:
com.siemens.polarion.rest.security.restApiToken.enabled=true
com.siemens.polarion.rest.enabled=true
#Optional
com.siemens.polarion.rest.swaggerUi.enabled=trueThen don’t forget to restart the polarion service.
We’ve now enabled the REST API and have a Polarion REST API token for the current session available if we call „window.getRestApiToken()“. This allows us to use the REST API in LiveReports and Form Extensions without having to create a REST API token first.
Find more information here: Enable REST API
Set up OpenAI Account
Before we can integrate an LLM, we need one. While there is a huge amount of LLMs available and you could host your own, it is very convenient that providers like OpenAI/Microsoft offer their models via API. This means that you can use a LLM that is hosted by Microsoft by simply paying a few cents instead of having to host your own and having to set up your own API and hosting.
Go to this website and sign up: https://platform.openai.com/docs/overview
Unfortunately this service isn’t free: https://openai.com/api/pricing/ and you’ll need a credit card to pay. But don’t be afraid, one dollar is enough to get you started and it will be enough to make about 100 requests with GPT-4o. If you use older models like GPT-3.5 (which was the model used by ChatGPT when it released), you won’t be able to spend the dollar even if you send 1000 requests. (At least for this use case – it depends on the amount of tokens used)
Getting the OpenAI API Key
In order to authenticate via the OpenAI API, you need to have an API Key. Therefore, go to „Dashboard“ –> „API Keys“ –> „Create new secret key“.
Save the API key, to use it in the form extension.
Integrating GPT-4o into Form Extension
Now we’ll come to the interesting part: asking the AI in our form extension to let it break down the user story into sub-tasks. In this chapter I’ll explain the code step by step. In the „complete code“ chapter you’ll find the whole source.
We start with creating a js function „gptRequestUserStoryBreakdown()“, which wil send the configuration and user story description to the AI:
#set($workItemTitle = $object.getTitle())
#set($workItemDescription = $object.getDescription().getContent().replaceAll("text/html: ",""))
<script>
// Variables
var openAiApiKey = "YOUR_API_KEY"
var contextWorkItemIdUserStory = "$workItemId";
var projectId = "$projectId";
var gptModelUserStoryBreakdown = 'gpt-4o'; //Other models: gpt-3.5-turbo
// GPT API request
var userStory = "$workItemTitle" + " - " + "$workItemDescription";
var formatUserStoryBreakdown = `The format of the answer should be a JSON structured like this:
{
"userStories": [
{
"title": "Example User Story Title",
"description": "As a User I want to [...] in order to [...]"
},
{
"title": "Example User Story Title",
"description": "As a User I want to [...] in order to [...]"
}
]
}`;
var configurationUserStoryBreakdown = 'You are an assistant for a requirements engineer and project manager. Please break down the following Requirements into User Stories.' + formatUserStoryBreakdown;
async function gptRequestUserStoryBreakdown(){
const requestOptions = {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${openAiApiKey}`
},
body: JSON.stringify({
"model": gptModelUserStoryBreakdown,
"messages": [{"role": "system", "content": configurationUserStoryBreakdown}, {"role": "user", "content": userStory}]
})
};
try {
const response = await fetch('https://api.openai.com/v1/chat/completions', requestOptions);
if (response.ok) {
const responseBody = await response.json();
console.log("API call successfull:", responseBody.choices[0].message.content);
contentGPT = responseBody.choices[0].message.content.replace(/```json|```/g, '');
} else {
console.log("API call failed:", response);
return null;
}
} catch (error) {
console.error("Error during API call:", error);
return null;
}
}
</script>
We’ll call this function by adding it as onclick function to a button below the table. But instead of calling it directly, we’ll wrap it into an additional function „generateSubTasks()“ that also manages the behaviour of the button (adding a loading symbol, deactivating it when finished) and also starts the parsing function when we get the answer from the AI back. Splitting this up is best practice to improve maintain- and reuseability.
<button class="ui button hicontrast small" onclick="generateSubTasks()">Generate User Stories via AI</button>
<script>
async function generateSubTasks(){
var clickedButton = event.target;
clickedButton.disabled = true;
clickedButton.innerHTML = '<i class="fa fa-spinner fa-spin"></i> Loading...';
var gptTaskBreakdown = await gptRequestUserStoryBreakdown();
clickedButton.innerHTML = 'Tasks were generated';
clickedButton.classList.add('ui-disabled');
}
</script>
When we get the response from the AI, we have to parse it and render the tasks in a table:
async function generateSubTasks(){
var clickedButton = event.target;
clickedButton.disabled = true;
clickedButton.innerHTML = '<i class="fa fa-spinner fa-spin"></i> Loading...';
var gptTaskBreakdownResponse = await gptRequestUserStoryBreakdown();
let tasksData = JSON.parse(gptTaskBreakdownResponse);
let tasks = tasksData.tasks;
tasks.forEach(taskItem => {
const { title, description } = taskItem;
addTableRow('-', title, description, '-');
});
clickedButton.innerHTML = 'Tasks were generated';
clickedButton.classList.add('ui-disabled');
}
function addTableRow(column1Data, column2Data, column3Data, column4Data) {
let table = document.getElementById("subTaskTable");
const newRow = table.insertRow();
const column1 = newRow.insertCell();
column1.textContent = column1Data;
const column2 = newRow.insertCell();
column2.textContent = column2Data;
const column3 = newRow.insertCell();
column3.textContent = column3Data;
const column4 = newRow.insertCell();
column4.textContent = column4Data;
const column5 = newRow.insertCell();
// Create a button element
var button = document.createElement("button");
button.innerHTML = "Create Work Item";
button.setAttribute('data-title', column2Data);
button.setAttribute('data-description', column3Data);
button.classList.add('ui', 'button', 'hicontrast', 'small');
// Add a click event listener to the button
button.addEventListener("click", function(){createWorkItem()});
// Add the button to the last cell in the new row
column5.appendChild(button);
}
Complete Code
Now there are still some small missing parts that I didn’t describe in detail: mostly the „createWorkItem()“ and „linkWorkItem()“ function that will utilize the Polarion REST API to quickly create the proposed tasks as linked work items. Instead I’ll just paste now the complete code. When you reuse it, you would have to add your own OpenAI API key to make the script work:
###############################################################################
## Title: AI GPT Task Breakdown
## Author: PolarionDude
## Date: 31.12.2024
## Version: 0.1
## Description: This extension uses AI to support the user in breaking down user stories into sub-tasks
##
###############################################################################
## Known-Issues:
## - Too many for production readyness
#######################
## Main
#######################
## Configuration and Context Variables
#set($taskWorkItemType = "task")
#set($workItemId = $object.getId())
#set($workItemTitle = $object.getTitle())
#set($workItemDescription = $object.getDescription().getContent().replaceAll("text/html: ",""))
#set($projectId = $object.getProjectId())
## Get current Work Item as Rendering API Object
#set($renderingWorkItem = $transaction.workItems().getBy().oldApiObject($object))
#set($linkedWorkItemsQuery = "SQL:(select WORKITEM.C_URI from WORKITEM inner join PROJECT on PROJECT.C_URI = WORKITEM.FK_URI_PROJECT inner join STRUCT_WORKITEM_LINKEDWORKITEMS as SWL on WORKITEM.C_URI = SWL.FK_URI_P_WORKITEM inner join WORKITEM as WORKITEM2 on WORKITEM2.C_URI = SWL.FK_WORKITEM where PROJECT.C_ID = '$projectId' AND WORKITEM.C_TYPE = '$taskWorkItemType' AND WORKITEM2.C_ID = '$workItemId')")
#set($linkedTasks = $trackerService.queryWorkItems($linkedWorkItemsQuery, "created"))
<table id="subTaskTable" class="polarion-rpw-table-content">
<tr class="polarion-rpw-table-header-row">
<th style="min-width: 50px;">ID</th>
<th>Title</th>
<th>Description</th>
<th style="min-width: 70px;">Status</th>
<th>Action</th>
</tr>
#foreach($linkedWorkItem in $linkedTasks)
<tr class="polarion-rpw-table-content-row">
<td style="min-width: 50px;">$!linkedWorkItem.getId()</td>
<td>$!linkedWorkItem.getTitle()</td>
<td>$transaction.workItems().getBy().oldApiObject($linkedWorkItem).fields().description().render().htmlFor().forFrame()</td>
<td style="min-width: 70px;">$!linkedWorkItem.getStatus().getName()</td>
<td>Already Existing</td>
</tr>
#end
</table>
<br>
<button class="ui button hicontrast small" onclick="generateSubTasks()">Generate User Stories via AI</button>
<script>
var openAiApiKey = "[YOUR_OPENAI_KEY]"
var contextWorkItemId = "$workItemId";
var projectId = "$projectId";
var gptModelUserStoryBreakdown = 'gpt-4o'; //Other models: gpt-3.5-turbo
// GPT API request
var userStory = "$workItemTitle" + " - " + "$workItemDescription";
var formatUserStoryBreakdown = `The format of the answer should be in JSON syntax structured like this. Don't add an additional extra opening brace anywhere that could crash the syntax:
{
"tasks": [
{
"title": "Example Task Title",
"description": "Example Description"
},
{
"title": "Example Task Title",
"description": "Example Description"
},
{
"title": "Example Task Title",
"description": "Example Description"
}
]
}`;
var configurationUserStoryBreakdown = 'You are an assistant for a requirements engineer and project manager. Please break down the following user story into sub tasks that have to be done by the developer and project team.' + formatUserStoryBreakdown;
var baseUrl = window.location.origin;
async function generateSubTasks(){
var clickedButton = event.target;
clickedButton.disabled = true;
clickedButton.innerHTML = '<i class="fa fa-spinner fa-spin"></i> Loading...';
var gptTaskBreakdownResponse = await gptRequestUserStoryBreakdown();
let tasksData = JSON.parse(gptTaskBreakdownResponse);
let tasks = tasksData.tasks;
tasks.forEach(taskItem => {
const { title, description } = taskItem;
addTableRow('-', title, description, '-');
});
clickedButton.innerHTML = 'Tasks were generated';
clickedButton.classList.add('ui-disabled');
}
async function gptRequestUserStoryBreakdown(){
const requestOptions = {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${openAiApiKey}`
},
body: JSON.stringify({
"model": gptModelUserStoryBreakdown,
"messages": [{"role": "system", "content": configurationUserStoryBreakdown}, {"role": "user", "content": userStory}]
})
};
try {
const response = await fetch('https://api.openai.com/v1/chat/completions', requestOptions);
if (response.ok) {
const responseBody = await response.json();
console.log("API call successfull:", responseBody.choices[0].message.content);
var gptResponse = responseBody.choices[0].message.content.replace(/```json|```/g, '');
return gptResponse;
} else {
console.log("API call failed:", response);
return null;
}
} catch (error) {
console.error("Error during API call:", error);
return null;
}
}
// This function adds a new row to the "subTaskTable"
function addTableRow(column1Data, column2Data, column3Data, column4Data) {
let table = document.getElementById("subTaskTable");
const newRow = table.insertRow();
const column1 = newRow.insertCell();
column1.textContent = column1Data;
const column2 = newRow.insertCell();
column2.textContent = column2Data;
const column3 = newRow.insertCell();
column3.textContent = column3Data;
const column4 = newRow.insertCell();
column4.textContent = column4Data;
const column5 = newRow.insertCell();
// Create a button element
var button = document.createElement("button");
button.innerHTML = "Create Work Item";
button.setAttribute('data-title', column2Data);
button.setAttribute('data-description', column3Data);
button.classList.add('ui', 'button', 'hicontrast', 'small');
// Add a click event listener to the button
button.addEventListener("click", function(){createWorkItem()});
// Add the button to the last cell in the new row
column5.appendChild(button);
}
async function createWorkItem(){
let clickedButton = event.target;
clickedButton.disabled = true;
clickedButton.innerHTML = '<i class="fa fa-spinner fa-spin"></i> Loading...';
let createWorkItemTitle = clickedButton.getAttribute("data-title");
let createWorkItemDescription = clickedButton.getAttribute("data-description");
const url = baseUrl + '/polarion/rest/v1/projects/' + projectId + '/workitems';
const requestBody = {
data: [
{
type: 'workitems',
attributes: {
type: 'task',
description: {
type: 'text/html',
value: createWorkItemDescription
},
title: createWorkItemTitle
}
}
]
};
try {
const response = await fetch(url, {
method: "POST",
headers: {
'X-Polarion-REST-Token': window.getRestApiToken(),
'Content-Type': 'application/json',
'Accept': 'application/json'
},
body: JSON.stringify(requestBody)
});
if (response.ok) {
const responseBody = await response.json();
console.log("Task created successfully:", responseBody);
clickedButton.innerHTML = "Task created successfully";
linkWorkItem(responseBody.data[0].id.split("/")[1]);
} else {
console.log("Failed to create task:", response.status);
}
} catch (error) {
console.error("Error creating task:", error);
}
}
async function linkWorkItem(sourceWorkItemId){
const urlLinkWorkItems = baseUrl + '/polarion/rest/v1/projects/' + projectId + '/workitems/' + sourceWorkItemId + '/linkedworkitems';
const requestBody = {
data: [
{
type: 'linkedworkitems',
attributes: {
role: 'relates_to'
},
relationships: {
workItem: {
data: {
type: "workitems",
id: projectId + '/' + contextWorkItemId
}
}
}
}
]
};
console.log(requestBody + urlLinkWorkItems);
try {
const response = await fetch(urlLinkWorkItems, {
method: "POST",
headers: {
'X-Polarion-REST-Token': window.getRestApiToken(),
'Content-Type': 'application/json',
'Accept': 'application/json'
},
body: JSON.stringify(requestBody)
});
if (response.ok) {
const responseBody = await response.json();
console.log("Link created successfully:", responseBody);
} else {
console.log("Failed to create link:", response.status);
}
} catch (error) {
console.error("Error creating link:", error);
}
}
function updateUserStoryVars() {
// Set or reset the JS variables with the new content
window.userStory = "$workItemTitle" + " - " + "$workItemDescription";;
}
updateUserStoryVars();
</script>
And voilà, we have our first simple AI integration working:
Improving Outputs
You might be right, if you say: that’s nice and all, but the tasks might not be correct and you would like to improve the accuracy of the AI output. This topic will be covered in the upcoming series. There is a wide variety of approaches to improve the outputs. There is a variety of approaches to improve output. To name a few: increasing context (and providing examples), improving prompts or training the models.
What to Consider for Production
What we’ve built now is not ready for production. To illustrate this, I’ve tried to visualize the architecture and explain it’s shortcomings.
What are the problems with this architecture?
- The token that is used to authenticate the user against OpenAI is visible to the user. This means, the user could „inspect“ the site, retrieve the token and use it for other AI applications. – Something that needs a moderate level of know-how. On the other hand only users that logged into Polarion in the first place will be able to retrieve it. Nevertheless, I would aruge that it’s a high risk. Because you can’t really find out, if someone is abusing your token and who does it. Instead you’ll just find a high bill for the AI services and wonder why that is. I’ll show you a solution for this in the „higher levels“.
- Sending potentially confidential data across the internet to the public OpenAI API. Although OpenAI says that they don’t use data sent via API as training data, all requests from around the world are using the same service. Hosted anywhere with unknown security measures. So I naturally distrust this with my confidential data. And Polarion usually contains highly sensitive data about our products. But great thing is: you can get your own dedicated LLM by Microsoft via Azure. Or you go with a completely different LLM, which can also be hosted by yourself or other hyperscalers like AWS.
- Your user needs direct access to the LLM. Where is the issue you might ask? And I would agree that it’s low risk to open up the AI service for anyone, because you need a token to use it. But especially if you enable your AI to access confidential data, you might want to add an additional security layer. Like IP whitelisting – only allowing certain applications (like Polarion) to access the AI. This would prevent anyone who can get their hands on a token to extract confidential information via the AI service directly.
- More… but I think these three are the main disadvantages right now.
Summary
Hopefully this post did inspire you to think about what else could be possible. And let me tell you: there is an incredible potential in these kinds of applications.
You will find the code example in my public GitHub repository. I would like to share these kinds of code snippets with you. Visit my PolarionDude GitHub.
Stay tuned and don’t forget to subscribe to the blog on the homepage, to prevent missing out on the upcoming posts of this series: