Audio & Video Sync
alphaTab can be synchronized with an external audio or video backing track. You can either use Guitar Pro 8 files with an audio track or synchronize alphaTab with an external media using the Media Sync Editor in the Playground.
At this point alphaTab cannot mix the synthesized audio and a backing track together, therefore only either audio will be heard. Using the synchronization information embedded in the data model, alphaTab can then place the cursor correctly as an external media is taking over the audio.
Audio Backing Track​
alphaTab has built-in support for audio backing tracks. To play alphaTab with an audio backing track instead of synthesized audio, following prerequisites have to be fulfilled:
- The data model ships the backing track audio file.
- The data model ships sync points which aligns the external audio track with the music notation.
- You enabled the backing track via either
PlayerMode.EnabledAutomaticorPlayerMode.EnabledBackingTrack. - Your platform (browser, operating system,...) has to support the embedded audio file. OGG Vorbis and MP3s are quite widely supported, and if used, there shouldn't be major problems.
To tackle 1. and 2. you can use the Media Sync Editor we provide on our Playground. Learn more about the editor here
Guitar Pro 8 Files​
Since Version 8 Guitar Pro provides built-in audio tracks with the possibility to synchronize the backing track with the music sheet. If the audio file is embedded into the GP file, alphaTab can load and use both the audio file and the sync point information.
As mentioned above, alphaTab can only play either the external audio file or the synthesized audio. Also we do not support changes in the audio (like adjusting the pitch).
Beside that you can load Guitar Pro 8 files and directly benefit from the enhanced sound experience.
Custom External Media Player​
alphaTab can be integrated with any external media system but it requires some implementation on the integrator side. To properly synchronize alphaTab and an external media source (audio or video) the alphaTab.synth.IExternalMediaHandler interface has to be implemented and provided to alphaTab.
Beside that, alphaTab has to be informed about the time updates on the external media (e.g. during playback and seeking). This update should happen at a rate smaller than two subsequent notes. 50ms updates have shown to work well on even fast songs, but to save power and battery you might want to handle this dynamically.
The following example illustrates a full integration with a HTML Media Element like <audio /> or <video />. This should be a good reference on how to implement the related sync with your media player.
Most things should be quite obvious like syncing volume, playback speed and providing the time and duration.
const audio = document.querySelector('#audio-element');
let updateTimer = 0;
const settings = new alphaTab.Settings();
// enable the external media control
settings.player.playerMode = alphaTab.PlayerMode.EnabledExternalMedia;
const api= new alphaTab.AlphaTabApi('#element', settings);
// Note: alphaTab works in milliseconds while HTML5 audio works with seconds
//
// handler for alphaTab -> audio tag updates
const handler: alphaTab.synth.IExternalMediaHandler = {
get backingTrackDuration() {
const duration = audio.duration;
return Number.isFinite(duration) ? duration * 1000 : 0;
},
get playbackRate() {
return audio.playbackRate;
},
set playbackRate(value) {
audio.playbackRate = value;
},
get masterVolume() {
return audio.volume;
},
set masterVolume(value) {
audio.volume = value;
},
seekTo(time) {
audio.currentTime = time / 1000;
},
play() {
audio.play();
},
pause() {
audio.pause();
}
};
(api.player!.output as alphaTab.synth.IExternalMediaSynthOutput).handler = handler;
//
// handlers for audio tag -> alphaTab updates
const onTimeUpdate = () => {
(api.player!.output as alphaTab.synth.IExternalMediaSynthOutput).updatePosition(
audio.currentTime * 1000
);
}
// time updates
audio.addEventListener('timeupdate', onTimeUpdate);
audio.addEventListener('seeked', onTimeUpdate);
audio.addEventListener('play', () => {
window.clearInterval(updateTimer);
newApi.play();
updateTimer = window.setInterval(onTimeUpdate, 50);
});
// state updates
audio.addEventListener('pause', () => {
newApi.pause();
window.clearInterval(updateTimer.current);
});
audio.addEventListener('ended', () => {
newApi.pause();
window.clearInterval(updateTimer.current);
});
audio.addEventListener('volumechange', () => {
newApi.masterVolume = audio.volume;
});
audio.addEventListener('ratechange', () => {
newApi.playbackSpeed = audio.playbackRate;
});
YouTube​
YouTube is a great counterpart to alphaTab to provide audio and video for your music sheet. Until now we decided to NOT ship a direct YouTube integration as there are quite some implications to it. Some key reasons behind this is:
- alphaTab is a cross-platform toolkit. While integrating a YouTube player in the web is easy, SDKs for Android or Desktop platforms are way more complex. We want to keep this on the integrator side.
- Looking at data protection laws like GDPR, users have to accept the load and integration of such external media. We want to be sure devs take proper care asking users for consent where required.
- The integration of the YouTube player requires further considerations in your user interface (sizing, where to place it, how to configure the YouTube player).
Nevertheless we want to give you some guidance on how to link alphaTab to YouTube. The following steps show how to use the YouTube Player API Reference for iframe Embeds together with alphaTab.
// assuming a <div id="youtube"></div> somewhere
//
// 1. load the YouTube IFrame API. we use promises to have a bit better control over the initialization sequence
const playerElement = document.getElementById('youtube');
const tag = document.createElement('script');
tag.src = "https://www.youtube.com/player_api";
playerElement.parentNode.insertBefore(tag, playerElement);
const youtubeApiReady = Promise.withResolvers();
window.onYouTubePlayerAPIReady = youtubeApiReady.resolve;
//
// 2. Already initialize alphaTab
const api = new alphaTab.AlphaTabApi('#alphaTab', {
core: {
file: 'my-synced-file.gp'
},
player: {
// set the external media mode
playerMode: alphaTab.PlayerMode.EnabledExternalMedia
}
});
window.at = at;
//
// 3. Wait for the youtube API to be ready
await youtubeApiReady.promise;
//
// 4. Setup the youtube player and wait for the video to be ready for playback
const youtubePlayerReady = Promise.withResolvers();
let currentTimeInterval = 0;
const player = new YT.Player(playerElement, {
height: '360',
width: '640',
videoId: 'your-video-id',
playerVars: { 'autoplay': 0 }, // we do not want autoplay
events: {
'onReady': (e) => {
youtubePlayerReady.resolve();
},
// when the player state changes we update alphatab accordingly.
'onStateChange': (e) => {
//
switch (e.data) {
case YT.PlayerState.PLAYING:
currentTimeInterval = window.setInterval(() => {
api.player.output.updatePosition(player.getCurrentTime() * 1000)
}, 50);
api.play();
break;
case YT.PlayerState.ENDED:
window.clearInterval(currentTimeInterval);
api.stop();
break;
case YT.PlayerState.PAUSED:
window.clearInterval(currentTimeInterval);
api.pause();
break;
default:
break;
}
},
'onPlaybackRateChange': (e) => {
api.playbackSpeed = e.data;
},
'onError': (e) => {
youtubePlayerReady.reject(e);
},
}
});
await youtubePlayerReady.promise;
//
// 5. Setup the handler to let alphaTab control the youtube player when needed
// Setup alphaTab with youtube handler
const alphaTabYoutubeHandler = {
get backingTrackDuration() {
return player.getDuration() * 1000;
},
get playbackRate() {
return player.getPlaybackRate();
},
set playbackRate(value) {
player.setPlaybackRate(value);
},
get masterVolume() {
return player.getVolume() / 100;
},
set masterVolume(value) {
player.setVolume(value * 100);
},
seekTo(time) {
player.seekTo(time / 1000);
},
play() {
player.playVideo();
},
pause() {
player.pauseVideo();
}
};
api.player.output.handler = alphaTabYoutubeHandler;
As you can see it is again just a matter of passing the right calls and values back and forth just like with an <audio /> element.
The YouTube IFrame API has a quite nasty behavior when it comes to seeking: For some strange reason seekTo will start the playback of the video
depending on the state. The docs say:
player.seekTo(seconds:Number, allowSeekAhead:Boolean):VoidSeeks to a specified time in the video. If the player is paused when the function is called, it will remain paused. If the function is called from another state (playing, video cued, etc.), the player will play the video.
alphaTab might initially do some seeking to the start position. This can cause the video to directly start playing which you might not want. To workaround this problem you could adjust the handler to check the player state and react accordingly.
let initialSeek = -1;
const alphaTabYoutubeHandler = {
get backingTrackDuration() {
return player.getDuration() * 1000;
},
get playbackRate() {
return player.getPlaybackRate();
},
set playbackRate(value) {
player.setPlaybackRate(value);
},
get masterVolume() {
return player.getVolume() / 100;
},
set masterVolume(value) {
player.setVolume(value * 100);
},
seekTo(time) {
if (
player.getPlayerState() !== YT.PlayerState.PAUSED &&
player.getPlayerState() !== YT.PlayerState.PLAYING
) {
initialSeek = value / 1000;
} else {
player.seekTo(value / 1000);
}
},
play() {
player.playVideo();
if (initialSeek >= 0) {
player.seekTo(initialSeek);
initialSeek = -1;
}
},
pause() {
player.pauseVideo();
}
};