Software Diagram and Improvements

To understand how to prove the feasibility of the idea using the Cyanite AI API, a high-level software diagram was created using the IT architecture sketching method.

To improve the created software diagram, tinkering with the Cyanite AI tool was carried out, alongside a literature study and gap analysis.

Software diagram

The functionality behind the concept can be explained in the following IT architecture sketch:

High-level software diagram of the concept

This high-level software diagram shows how the tool would suggest music results once the user provides input to the tool. Using the usability testing insights from the previous deliverable, two major improvements were made.

Improvement #1: Prompt guidance

Filmmakers might need guidance or examples of what kind of prompts they can enter to get suitable music results. In some cases they might know what music they want, while in others they would prefer the tool to recommend it for them.

A text prompt is a powerful feature that has the potential to provide desired results. However, usually such user freedom also brings design responsibilities. If users lack examples or guidance on how to effectively use the feature, they may not be able to make the most out of it and achieve satisfactory results.

After studying some of the main elements that comprise a film, a template was developed to guide filmmakers in describing their film project using the following elements:

• project type - be it a short film, scene, video, etc.

• context - what the film is about

• theme - the filmmaker's perspective on the central idea of the film. For example, if the central idea is “love”, the theme can be "love is blind"

• tone - the attitude of the filmmaker or the attitude conveyed by the film about the theme (e.g. "comedic", "dramatic")

The full desk research is available below.

Each of these elements could be described in an extensive way but filmmakers can usually find a short phrase to describe each one, and incorporate them into a short text of the form:

For example:

Additionally, if filmmakers know what music they want, they can follow a template like:

Impact: This improvement makes the concept more accessible for filmmakers as they can be guided by examples of how to provide a prompt.

Improvement #2: Adding openAI API

It was discovered that Cyanite AI’s “search by text” feature sometimes performs better when provided with more direct musical description, rather than any free text prompt.

The following example demonstrates the findings:

Therefore, Cyanite AI’s “search by text” feature empowers the input of any desired text but in some cases the results are not very accurate which contradicts the vision for the advanced music discovery tool. Consequently, measures needed to be taken to address this issue and ensure more precise results.

An opportunity was found in using ChatGPT to interpret the free text prompt into a more direct musical description.

Example:

Interpreting a free text prompt into direct musical descriptions using ChatGPT. Such musical descriptions would provide overall better results in Cyanite AI.

To add this finding to the concept, an additional call to the API of OpenAI (the company behind ChatGPT) needs to be done. The addressed improvement can be placed in the IT architecture sketch in the following way:

High-level software diagram of the concept after the improvements

Impact: This improvement aims to make the concept more bulletproof so that more accurate music suggestions are provided based on the given prompt.

After creating an IT architecture sketch to understand how the concept would work from the software side, two improvements were made to ensure better music results.

Having improved the concept, the next step was to incorporate all these findings to make wireframes of the UI, design and technical prototypes.