Seek is not accurate with DRM content.
I seek to the specific SMPTE 00:01:35:15 timecode (95.742 in seconds). After the Bitmovin player Seeked event getCurrentTime returns this time 95.957333 instead of 95.742. But seek target was 95.742.
Hi @anatolii.ibrahimov , thanks for sharing the observation here. Can you please help us understand the use case a bit more. Are you looking for frame accurate seeking?
Looking at your observations, it seems that the seeked to position(95.957333) is 1 frame ahead of the requested seek position(95.742). But it is not clear how is the seek position(95.742) calculated from SMPTE timestamp (00:01:35:15). The fraction part .742 seems to be calculated from the frame number part(15) of SMPTE timestamp. Can you please elaborate on this calculation?
Hi @lucky.goyal! Yes, I’m looking for frame accurate seeking.
I integrated this code example to our project. demos/script.js at 2512bf839410b662a1a72a3556de5afaa522c0f0 · bitmovin/demos · GitHub
export default class SmpteTimestamp {
frame;
seconds;
minutes;
hours;
assetDescription;
constructor(smtpeTimestamp: any, assetDescription: any) {
this.assetDescription = assetDescription;
if (smtpeTimestamp && Number.isFinite(smtpeTimestamp)) {
let smpteValue = smtpeTimestamp;
this.frame = smpteValue % 100;
smpteValue = Math.floor(smpteValue / 100);
this.seconds = smpteValue % 100;
smpteValue = Math.floor(smpteValue / 100);
this.minutes = smpteValue % 100;
this.hours = Math.floor(smpteValue / 100);
} else if (
smtpeTimestamp &&
SmpteTimestamp.validateTimeStamp(smtpeTimestamp, assetDescription.framesPerSecond)
) {
const [hours, minutes, seconds, frame] = smtpeTimestamp.split(':').map(Number);
this.hours = hours;
this.minutes = minutes;
this.seconds = seconds;
this.frame = frame;
} else {
this.hours = 0;
this.minutes = 0;
this.seconds = 0;
this.frame = 0;
}
}
static validateTimeStamp = (smtpeTimestamp: any, framesPerSecond: any) => {
const isValidSMPTETimeCode = /^(?:(?:[0-1][0-9]|[0-2][0-3]):)(?:[0-5][0-9]:){2}(?:[0-6][0-9])$/;
if (!isValidSMPTETimeCode.test(smtpeTimestamp)) {
throw new Error(`${smtpeTimestamp} does not match a SMPTE timecode HH:MM:SS:FF`);
}
if (Number(smtpeTimestamp.split(':')[3]) >= framesPerSecond) {
throw new Error(`Frame Number in SMPTE is higher than FPS: ${smtpeTimestamp}`);
}
return true;
};
static padNum = (num: any) => (num < 10 ? `0${num}` : String(num));
static fitIntoRange = (toFit: any, range: any) => {
let overflow = 0;
if (toFit < 0) {
while (toFit < 0) {
overflow -= 1;
toFit += range;
}
} else if (toFit >= range) {
while (toFit >= range) {
overflow += 1;
toFit -= range;
}
}
return [toFit, overflow];
};
static fromTime = (timestamp: any, assetDesc: any) => {
// to get to the start of the actual frame... use this
let tmp = timestamp;
const retVal = new SmpteTimestamp(null, assetDesc);
retVal.hours = Math.floor(tmp / 3600);
tmp -= retVal.hours * 3600;
retVal.minutes = Math.floor(tmp / 60);
tmp -= retVal.minutes * 60;
retVal.seconds = Math.floor(tmp);
tmp -= retVal.seconds;
retVal.frame = Math.floor(tmp / assetDesc.frameDuration);
return retVal;
};
static fromTimeWithAdjustments = (timestamp: any, assetDesc: any) => {
const time = timestamp / assetDesc.adjustmentFactor;
const smtpe = SmpteTimestamp.fromTime(time, assetDesc);
if (assetDesc.framesDroppedAtFullMinute > 0) {
let numMinutesWithDroppedFrames = smtpe.minutes + (smtpe.hours * 60);
// no frames dropped at every 10 minutes
numMinutesWithDroppedFrames -= Math.floor(numMinutesWithDroppedFrames / 10);
const framesToAdd = numMinutesWithDroppedFrames * assetDesc.framesDroppedAtFullMinute;
const minutesBefore = smtpe.minutes;
smtpe.addFrame(framesToAdd, false);
if (smtpe.minutes % 10 !== 0 && minutesBefore !== smtpe.minutes) {
smtpe.addFrame(assetDesc.framesDroppedAtFullMinute, false);
}
}
return smtpe;
};
toString = () => {
return `${SmpteTimestamp.padNum(this.hours)}:${SmpteTimestamp.padNum(this.minutes)}:${SmpteTimestamp.padNum(
this.seconds
)}:${SmpteTimestamp.padNum(this.frame)}`;
};
toTime = () => {
const { frameDuration } = this.assetDescription;
const timeInSeconds = this.hours * 3600 + this.minutes * 60 + this.seconds;
const frameTime = this.frame * frameDuration;
return timeInSeconds + frameTime;
};
addHour = (hoursToAdd: any) => {
this.hours += hoursToAdd;
if (this.hours < 0) {
console.log('Cannot go further back');
this.hours = 0;
this.minutes = 0;
this.seconds = 0;
this.frame = 0;
}
};
addMinute = (minutesToAdd: any) => {
const [toFit, overflow] = SmpteTimestamp.fitIntoRange(this.minutes, 60);
this.minutes += minutesToAdd;
this.minutes = toFit;
if (overflow !== 0) {
this.addHour(overflow);
}
};
addSeconds = (secondsToAdd: any) => {
const [toFit, overflow] = SmpteTimestamp.fitIntoRange(this.seconds, 60);
this.seconds += secondsToAdd;
this.seconds = toFit;
if (overflow !== 0) {
this.addMinute(overflow);
}
};
addFrame = (framesToAdd: any, fixFrameHoles: any) => {
if (fixFrameHoles === undefined) { fixFrameHoles = true; }
const [toFit, overflow] = SmpteTimestamp.fitIntoRange(this.frame, Math.ceil(this.assetDescription.framesPerSecond));
this.frame += framesToAdd;
this.frame = toFit;
if (overflow !== 0) {
this.addSeconds(overflow);
}
// make sure we dont step into a frame hole
if (fixFrameHoles && this.assetDescription.framesDroppedAtFullMinute > 0 && this.minutes % 10 !== 0) {
if (framesToAdd > 0 && this.seconds === 0) {
this.addFrame(this.assetDescription.framesDroppedAtFullMinute, false);
}
}
};
toAdjustedTime = () => {
const { adjustmentFactor, framesDroppedAtFullMinute, offsetToMidFrame } = this.assetDescription;
if (framesDroppedAtFullMinute > 0) {
const totalMinutes = this.hours * 60 + this.minutes;
let framesToAdd = totalMinutes - Math.floor(totalMinutes / 10);
framesToAdd *= framesDroppedAtFullMinute;
this.addFrame(-framesToAdd, false);
}
let targetTime = this.toTime() * adjustmentFactor;
targetTime += offsetToMidFrame;
targetTime = Math.floor(targetTime * 1000) / 1000;
return targetTime;
};
}
const currentTime = playerInstance?.current?.getCurrentTime();
const currentSMPTETime = SmpteTimestamp.fromTimeWithAdjustments(currentTime, assetDescription).toString();
const seekToSMPTE = (smpteString: string) => {
try {
console.log(smpteString);
const smpte = new SmpteTimestamp(smpteString, assetDescription);
console.log(smpte);
const debugSmpte = smpte.toString();
const targetTime = smpte.toAdjustedTime();
console.log(`Seeking to SMTPE: ${debugSmpte}, calculated Time: ${targetTime}`);
playerInstance?.current?.seek(targetTime);
} catch (error) {
console.error(`Error during converting smtpe to time: ${error}`);
throw error;
}
};
@lucky.goyal also it’s not 1 frame ahead.
The api is returning a value greater than the target on 5 frames.
Timecode of nearest frame to 95.742 is 00:01:35:18 => 95.750
Timecode based on returned value from the Bitmovin player is 00:01:35:23 => 95.958
Hi @anatolii.ibrahimov , thanks for sharing that you are using Bitmovin’s frame accurate seek demo. You are also right about the frame difference between seek request and actual seeked point.
I will look into it further and get back with feedback.
Meanwhile, can you please also share the use case behind the frame accurate seeking requirement. General use case we have come across is content post processing which is normally done on non-DRM content. Knowing the use case will be helpful in understanding the requirements better.
Hi @lucky.goyal!
Hence we are working only with DRM content. We need to be able to jump to the specific timecode (SMPTE, frames, milliseconds). It’s extremely important to have accurate seeking as we review content inside the player. After reviewing the content, it can be approved, rejected, etc.
Hi @anatolii.ibrahimov , I tested a couple of sample Widevine DRM assets on Chrome browser and found the frame accurate seeking demo to work fine. One of the DRM asset that I tested is below.
source: {
dash: "https://bitmovin-a.akamaihd.net/content/art-of-motion_drm/mpds/11331.mpd",
drm: {
playready: {},
widevine: {
LA_URL: "https://cwip-shaka-proxy.appspot.com/no_auth",
},
},
}
Can you please share a sample DRM asset where you observe the mismatch between target seek position and seeked to position? Please feel free to DM the source configuration.
Hi @anatolii.ibrahimov , checking in to see if you are able to test above DRM sample. I am also awaiting information on your DRM test asset to investigate the behaviour. Please do not hesitate to share if you have further questions/update on this topic.
Hi @lucky.goyal! Unfortunately we are not owners of the content so I can’t share it.
The video has 23.976 frame rate.
And we tested it with no sound in manifest. Only without audio it seems like seeking was accurate.
thanks for the update @anatolii.ibrahimov . I am afraid, it may not be possible to investigate further without having a way to replicate the behaviour.
What I can suggest at the moment is
- check multiple assets to asses if the behaviour is limited to specific assets or across all your assets.
- As the issue seems to happen only when audio is also present, compare the audio with working asset.
- Check if the issue can also be replicated with clear(non-DRM) copy of same asset. This helps to rule in/out DRM encryption.
Please feel free to reach out again if you are able to share a sample asset demonstrating the behaviour.