Music v2: Create, Control, and Remix AI Music

Peter Bubenik ยท Elevenlabs Research ยท ยท Source
Music v2: Create, Control, and Remix AI Music

Step-by-Step Lesson

Step 1: What Is Music v2?

Music v2 is ElevenLabs' upgraded AI music generation model.

Think of it like upgrading from a basic recording studio to a professional one. The same idea, but dramatically better results.

Key improvements over Music v1:

AreaImprovement
VocalsBetter quality and complexity
InstrumentationMore realistic and varied
ArrangementMore musically coherent
Language SupportMore reliable multilingual output

๐Ÿ’ก Simple analogy: If Music v1 was a skilled amateur musician, Music v2 is a professional studio musician who speaks multiple languages.


Step 2: What Can Music v2 Actually Do?

Music v2 introduces four major new capabilities:

๐ŸŽญ Capability 1: Genre Flexibility

A single song can now:

  • Shift from opera โ†’ heavy metal โ†’ back
  • Handle fast rap and dense lyrical delivery
  • Embed non-musical sound effects within a track

All without losing musical coherence.

๐Ÿ’ก Think of it like: A musician who can seamlessly switch styles mid-performance without the song falling apart.


โœ‚๏ธ Capability 2: Improved Inpainting

Inpainting = regenerating only a specific section of a track.

How it works:

  1. You generate a full song
  2. You select one section (e.g., the bridge)
  3. You regenerate only that part
  4. Everything else stays exactly the same

๐Ÿ’ก Analogy: Like using Photoshop to edit one small area of an image without touching the rest.


๐Ÿ—๏ธ Capability 3: Full Song Structure Building

Previously, AI music tools generated short clips.

Now Music v2 lets you build a complete song section by section:

Intro โ†’ Verse โ†’ Chorus โ†’ Bridge โ†’ Outro

The model maintains structure and continuity throughout the entire song.

๐Ÿ’ก Think of it like: Building with LEGO blocks โ€” each piece connects logically to the next.


๐ŸŒ Capability 4: Multilingual Support

  • Lyrics, vocals, and arrangements now perform more reliably in the language you write in
  • Meaningful improvements across a growing list of supported languages

Step 3: The Three Platforms โ€” Who Is Each For?

Music v2 powers three distinct platforms, each designed for a different user:

๐ŸŽต Platform 1: ElevenMusic

Who it's forMusicians and creators
What it isA creative studio
What you can doCreate from lyrics/mood, develop full compositions, remix tracks
AnalogyYour personal recording studio

๐Ÿ’ป Platform 2: ElevenAPI

Who it's forDevelopers
What it isDirect model access via API
What you can doGenerate, inpaint, reference-match programmatically
AnalogyA music engine you plug into your own app

๐Ÿ“ข Platform 3: ElevenCreative

Who it's forBrands and content teams
What it isLicensed music generation at scale
What you can doBrief by mood, genre, tempo, brand voice
Key benefitNo sync fees. No clearance delays.
AnalogyA creative music director on demand

Step 4: Pricing & Availability

๐Ÿ’ฐ Price Reductions

  • ElevenAPI: Up to 50% cheaper
  • ElevenCreative: Up to 40% cheaper

๐Ÿ“… Availability

PlatformStatus
ElevenMusicโœ… Available now
ElevenCreativeโœ… Available now
ElevenAPI๐Ÿ”œ Coming soon (early access available)

โš–๏ธ Rights & Licensing

  • Built in partnership with artists, labels, and publishers
  • Cleared for commercial use
  • Every track generated is yours to use

Quick Recap: The Big Picture

Music v2
    โ”‚
    โ”œโ”€โ”€ Better: Vocals, Instrumentation, Arrangement, Multilingual
    โ”‚
    โ”œโ”€โ”€ New Capabilities:
    โ”‚       โ”œโ”€โ”€ Genre flexibility (opera โ†’ metal)
    โ”‚       โ”œโ”€โ”€ Inpainting (edit one section only)
    โ”‚       โ”œโ”€โ”€ Full song structure building
    โ”‚       โ””โ”€โ”€ Multilingual lyrics/vocals
    โ”‚
    โ””โ”€โ”€ Three Platforms:
            โ”œโ”€โ”€ ElevenMusic โ†’ Creators
            โ”œโ”€โ”€ ElevenAPI โ†’ Developers
            โ””โ”€โ”€ ElevenCreative โ†’ Brands

โœ… Check Your Understanding

Test yourself with these questions:

  1. What is inpainting and why is it useful?
  2. Which platform would a mobile app developer use to add background music to their app?
  3. What is the key advantage of ElevenCreative for brands over traditional music licensing?
  4. Name two vocal/musical improvements Music v2 offers over Music v1.

๐ŸŽฏ If you can answer all four confidently, you've mastered this material!

More to study