There is a video on YouTube right now where the author claims that the Sony Alpha cameras don’t record correctly internally when shooting S-Log2 or S-Log3. The information contained in this video is highly miss-leading and the conclusion that the problem is with the way Sony record internally is incorrect. There really isn’t anything wrong with the way Sony do their recordings. Neither is there anything wrong with the HDMI output. While centered around the Alpha cameras the information below is also important for anyone that records S-Log2 or S-log3 externally with any other camera.
Some background: Within the video world there are 2 primary ranges that can be used to record a video signal.
Legal Range uses code value 16 for black and code value 235 for white (anything above CV235 is classed as a super-white and these can still be recorded but considered to be beyond 100%).
Full or Data Range uses code value 0 for black and code value 255 for white or 100%.
Most cameras and most video systems are based on legal range. ProRes recordings are almost always legal range. Most Sony cameras use legal range and do include super-whites for some of the curves such as Cinegammas or Hypergammas to gain a bit more dynamic range. The vast majority of video recordings use legal range. So most software defaults to legal range.
But very, very importantly – S-log2 and S-log is always full/data range.
Most of the time this doesn’t cause any issues. When you record internally in the camera the internal recordings have metadata that tells the playback, editing or grading software that the S-Log files have been recorded using full range. Because of this metadata the software will play the files back and process them at the correct levels. However if you record the S-Log with an external recorder the recorder doesn’t always know that what it is getting is full range and not legal range, it just records it, as it is, exactly as it comes out of the camera. That then causes a problem later on because the externally recorded file doesn’t have the right metadata to ensure that the full range S-Log material is handled correctly and most software will default to legal range if it knows no different.
Lets have a look at what happens when you import an internally recorded S-Log2 .mp4 file from a Sony A7S into Adobe Premiere:
A few things to note here. One is Adobe’s somewhat funky scopes where the 8 bit code values don’t line up with the normally used IRE values used for video productions. Normally 8 bit code value 235 would be 100IRE or 100%, but for some reason Adobe have code value 255 lined up with 100%. My suspicion is that the scope % scale is not video % or IRE but instead RGB%. This is really confusing. A further complication is that Adobe have code value 0 as black, again, I think, but am not sure that this is RGB code value 0. In the world of video Black should be code value 16. But the scopes appear to work such that 0 is black and that 100 is full scale video out. Anything above 100 and below 0 will be clipped in any file you render out.
Looking at the scopes in the screen grab above, the top step on the grey scale chart is around code value 252. That is the code value you would expect it to be, that lines up just nicely with where the peak of an S-Log2 recording should be. This all looks correct, nothing goes above 100 or below 0 so nothing will be clipped.
So now lets look at an external ProRes recording, recorded at exactly the same time as the internal recording and see what Premier does with that:
OK, so we can see straight away something isn’t quite right here. In an 8 bit recording it should be impossible to have a code value higher that 255, but the scopes are suggesting that the recording has a peak code value of something around CV275. That is impossible, so alarm bells should be ringing. Something is not quite right here. In addition the S-Log2 appears to be going above 100, so that means if I were to simply export this as a new file, the top of the recording will be clipped and it won’t match the original. This is very clearly not right.
Now lets take a look at what happens in Adobe Premiere when you apply Sony’s standard S-Log2 to Rec-709 LUT to a correctly exposed internal recording:
This all looks good and as expected. Blacks are sitting down just above the 0 line (which I think we can safely assume is black) and the whites of the picture are around code value 230 or 90, whatever that means. But they are certainly nice and bright and are not in the range that will be clipped. So I can believe this as being more or less correct and as expected.
So next I’m going to add the same standard LUT to the external recording to see what happens.
OK, this is clearly not right. Our blacks now go below the 0 line and they look clipped. The highlights don’t look totally out of place, but clearly there is something going very, very wrong when we this normal LUT to this correctly exposed external recording. There is no way our blacks should be going below zero and they look crushed/clipped. The internal recording didn’t behave like this. So what is going on with the external recording?
To try and figure this out lets take a look at the same files in DaVinci Resolve. For a start I trust the scopes in Resolve much more and it is a far better programme for managing different types of files. First we will look at the internal S-Log2 recording:
Once again the levels of the internal S-Log2 recordings look absolutely fine. Our peak is around code value 1010 which would be 252 in 8 bit. Right where the brightest bits of an S-log2 file should be. Now lets take a look at the external recording.
If you compare the two screen grabs above you can see that the levels are exactly the same. Our peak level is around CV1010/CV252, just where it should be and the blacks look the same also. The internal and external recordings have the same levels and look the same. There is no difference (other then perhaps less compression and fewer artefacts in the ProRes file). There is nothing wrong with either of these recordings and certainly nothing wrong with the way Sony record S-Log2 internally. This is absolutely what I expect to see.
BUT – I’ve been a little bit sneaky here. As I knew that the external recording was a full range recording I told DaVinci Resolve to treat it as a full range recording. In the media bin I right clicked on the clip and under “clip attributes” I changed the input range from “auto” to “full”. If you don’t do this DaVinci Resolve will assume the ProRes file to be legal range and it will scale the clip incorrectly in the same way as Premiere does. But if you tell Resolve the clip is full range then it is handled correctly.
This is what it looks like if you allow Resolve to guess at what range the S-Log2 full range clip is by leaving the input range setting to “auto”:
In the above image we can see how in Resolve the clip becomes clipped because in a legal range recording anything over CV235/CV940 would be an illegal super white. Resolve is scaling the clip and pushing anything in the original file that was above CV235/CV940 off the top of the scale. The scaling is incorrect because Resolve doesn’t know the clip is supposed to be full range and therefore not scaled. If we compare this to what Premiere did with the external recording it’s actually very similar. Premiere also scaled the clip, only Premiere will show all those “illegal” levels above it’s 100 line instead of clipping then as Resolve does. That’s why Premiere can have those “impossible” 8 bit code values going up to CV275.
Just to be complete here, I did also test the internal .mp4 recordings in Resolve switching between “auto” and “full” range and in both cases the levels stayed exactly the same. This shows that Resolve is correctly handling the internally record full range S-Log as full range.
What about if you add a LUT? Well you MUST tell Resolve to treat the S-Log2 ProRes clip as a full range clip otherwise the LUT will not be right, if your footage is S-Log3 you also have to tell Resolve that it is full range:
CONCLUSIONS:
Both the internal and external recordings are actually exactly the same. Both have the same levels, both use FULL range. There is absolutely nothing wrong with Sony’s internal recordings. The problem stems from the way most software will assume that the ProRes files are legal range. But if it’s an S-Log2 or S-Log3 recording it will in fact be full (data) range. Handling a full range clip as legal range means that highlights will be too high/bright or clipped and blacks will be crushed. So it’s really important that your software handles the footage correctly. If you are shooting using S-Log3 this problem is harder to spot as S-Log3 has a peak recording level that is well with the legal range, so you often won’t realise it’s being scaled incorrectly as it won’t necessarily look clip. If you use LUT’s and your ProRes clips look crushed or highlights look clipped you need to check that the input scaling is correct. It’s really important to get this right.
Why is there no difference between the levels when you shoot with a Cinegamma? Well when you shoot with a cinegamma the internal recordings are legal range so the internal recordings get treated as legal range and so do the external recordings, so they don’t appear to be different (In the YouTube video that led to this post the author discovers that if you record with a normal profile first and then switch to a log profile while recording the internal and external files will match. But this is because now the internal recording has the incorrect metadata, so it too gets scaled incorrectly, so both the internal and external files are now wrong – but the the same).
Once again: There is nothing wrong with the internal recordings. The problem is with the way the external recordings are being handled. The external recordings haven’t been recorded incorrectly, they have been recorded as they should be. The problem is the edit software is incorrectly interpreting the external recordings. The external recordings don’t have the necessary metadata to mark the files as full range because the recorder is external to the camera and doesn’t know what it’s being sent by the camera. This is a common problem when using external recorders.
What can we do in Premiere to make Premiere work right with these files?
You don’t need to do anything in Premiere for the internal .mp4 recordings. They are handled correctly but Premiere isn’t handling the full/data range ProRes files correctly.
My approach for this has always been to use the legacy fast color corrector filter to transform the input range to the required output range. If you apply the fast color corrector filter to a clip you can use the input and output level sliders to set the input and output range. In this case we need to set the output black level to CV16 (as that is legal range black) and we need to set output white to CV235 to match legal range white. If you do this you will then see that the external recording appears to have almost exactly the same values as the internal recording. However there is some non-linearity in the transform, it’s not quite perfect. So if anyone knows of a better way to do this do please let me know.
Now when you apply a LUT the picture and the levels are more or less what you would expect and almost identical to the internal recordings. I say almost because there is a slight hue shift. I don’t know where the hue shift comes from. In Resolve the internal and external recordings look pretty much identical and there is no hue shift. In Premiere they are not quite the same. The hue is slightly different and I don’t know why. My recommendation – use Resolve, it’s so much better for anything that needs any form of grading or color correction.
Great posting!
Great information Alister!! If you right click on the Lumetri Color panel, there is an option to select High Dynamic Range. It looks like the scope displays correctly. But, I am not sure if Lumetri is processing as 8 bit or 32 bit Floating??? Does this appear to be the correct workflow?
When shooting on the FS5 mk2 and PP1 (venice) it appears to grade more accurately with HDR selected.
Thanks for covering this topic. I’m the one who made the video you mentioned. I like your dive into the topic, but unfortunately it still doesn’t fix the changing colours as you said yourself with, “I say almost because there is a slight hue shift.” I can’t seem to solve this part either. Outside of using other software like you mentioned. This is the main reason why I suggest doing the changing-profiles-while-recording hack, even if both are incorrect, at least they match incorrectly together and provide for a quicker grade. (At least for Premiere users.)
Lastly, your Fast Color Corrector Filter idea is a decent solution, but I feel the Full to Legal range LUTs included in Premiere actually do a better job and might be slightly faster.
Good article! Cheers.
I see you have now finally changed the title of your video but sadly the damage has been done and a lot of people are now wondering if there is something wrong with their cameras. I tried to get you to look at this not long after you posted the video pointing out the glaringly obviously incorrect 8 bit code values, as have other people in the comments. But until now you have ignored everyone. Your statement that legal range reduces the ability to reproduce absolute blacks or whites is just nonsense. The recordings are not being clamped. Clamping means ensuring that the video signal does not have a DC offset and that the black level sits at zero. It’s nothing to do with the range of a video signal, only the DC or black offset.
Changing profiles while recording approach has to be one of the most awkward ways possible to approach this, it’s a bodge. Plus as Premiere will then incorrectly interpret both files incorrectly anyone wishing to use 3rd party LUT’s has a major problem. Better to fix the problem properly than to break both files. It shouldn’t be necessary to use a LUT to change the range from data to video. If the levels of the files were correctly handled then the files would look the same, but obviously they are still not quite right. Premiere is probably one of the worst pieces of software for grading on the market right now, yet it’s also one of the most expensive.
It’s unfortunate that you’re choosing to be dramatic instead of kind. I feel I was nicer in my reply. But hey, it’s your blog. I’m sorry if you feel I didn’t give your comment adequate consideration. I get a lot of them and I do my best to consider and reply as I can.
I’m just now seeing your idea to test middle grey for 32%, which is a great idea. YouTube is much better for notifying of initial comments than it is for replies. It’s difficult for me to find follow-ups unless I comb through every comment and at hundreds per day, it’s just not possible. I don’t “ignore” people. Sorry that from your angle it seemed like I was. I can appreciate why you felt that way.
Lastly, it’s important to realize that I did change my title, and I do accept corrections and feedback, and I do pin comments and share other people’s arguments–including your article, which is now pinned to the video. Instead you’re focusing on the negative and talking about “damage being done”. Let’s strive to be constructive and kind.
I don’t want this to turn into some big debate. I like your insight. I like your article. I pinned it to my video and will point people here for clarification. I’m done with this now and wish you the best!
You say you accept corrections and feedback yet you clearly have no intention of removing a video that still implies a problem with the way the camera records S-Log. The video starts with the comment that there is something peculiar about the way the camera records log. There is nothing peculiar at all, that is exactly how it is supposed to be. All the stuff around the 5 min mark implies the camera is not behaving correctly, but it is behaving correctly, you just don’t understand what is going on.
You say you accept corrections but you’re going to leave online a video that contains a whole bunch of errors and in the end recommends switching picture profiles mid shot. Something that will result in incorrectly recorded internal log recordings with the wrong metadata. How is that going to effect someone that uses Resolve or FCP? They will end up with problems if they follow your advice. What about people that use 3rd party LUT’s in Premiere? They too will end up with problems if they follow your advice. The problem is with the way Premiere handles the files, yet although you go through all the different ways to shift the range in Premiere everything in the video points at the camera being the cause of the problem because according to you, it’s recording a reduced or clamped range. But even though you now know this to be incorrect, you’ve chosen to leave the video online.
I guess we come from different generations. In my world it’s about always giving the best possible advice and to not miss-lead. To be accurate and correct. I am a human and realise that as a human I will make mistakes from time to time. When you have 1000’s of people that take everything you publish at face value you have a responsibility to ensure that errors are corrected asap. If that means re-writing, re-editing or removing a video or article then that’s what needs to happen. If you’ve said something is broken, defective or similar, then you need to own up to the mistake and show the manufacturer the respect they deserve with a correction and apology. Not something hidden in the comments, but somewhere where everyone that’s already seen the offending publication will see it. Not everyone bothers to reads the comments. I couldn’t leave a video with known bad advice or known incorrect assumptions online, that’s not my style.
And you know what – when someone corrects me, sure, it affects my feelings. But you just suck it up, learn from it, put it right and move on. If everyone is too scared to never offend or upset anyone and every conversation has to be “kind” then we are all doomed to be surrounded by bad information as everyone will be too scared to point out errors just in case they hurt someones feelings.
Once again Alister your in depth testing and explanations are very helpful! Great post.
I got a7III and noticed this, weird things when grading S-log2 footage but it was that “video levels”. When is chose “full range” in attributes everything graded just nicely. But the thing is i need to do that on every file and that is annoying. Im using Resolve 16.
You can select all of the files in the media room and change them all at the same time.
Hi,
I’m trying to capturing raw PP7, PP8, PP9 stills on the A7III for vfx reference shoot purposes. I then process them as tiffs files through the Sony imaging edge software . For the life of me I can’t get an accurate match, shooting a macbeth chart under daylight. Using Sony’s own LUTs (for slog2, slog3 slog3cine to rec709) in resolve and Nuke I can’t get a match to the ideal macbeth chart. The colors signficantly different. I was wondering if you had any luck getting a proper match with raw stills as opposed to the 8bit internal recording?
Log is always recorded using full/data range. Are you keeping everything as full range? If you don’t the LUT’s won’t work correctly.
This is an example of why I’m always telling people new to video to be careful where they get their information from, because there is so much misleading, inappropriate, and downright wrong advice regarding cameras and cinematography on the internet. Thanks for posting this article Alister, I have referred people to it on more than one occasion.
I’m so thankful to have found your site, Alister!
I’m currently shooting on the A7Siii with the Ninja V and it’s all super frustrating trying to find quality and accurate information.
I’m interested to know if you have time……If I was to NOT use the Ninja V’s “Legalise” option with my Slog3 footage, using the Waveform monitor, where should Middle Grey and White sit respectively if not 41IRE and 60IRE?
Is there any downside to using the Legalise option? And does that automatically add the correct metadata that was missing previously to the footage for resolve when recording ProRes?
Also, I’ve been trying to find information/graphs/tables about usable stops above and below middle grey if we are to expose it to 41%? I’ve purchased a Sekonic L-478D and was thinking it might be easier to manually input a custom camera profile based off of this information as I don’t have a spot meter for reflected reading and there’s so little information around to support this.
Once again, thanks so much for your dedication and hard work in putting this together. You’ve made things a little less foggy when it comes to how confusing IRE vs Data Levels and Full vs Legal Levels are.
James.
If you do not use the legalise function then the recording happens at the correct levels so you use 41/61 IRE. This gives a range of -8 and +6 stops.
Years later and this is still very much an issue!
(as in, NLEs/grading packages/external recorders not flagging or reading flags correctly)
Over a year ago I randomly heard about some argument between you both, both being people who I get information from. I didn’t know what it was about and was curious, but I never found the thread. I somehow came across this article and now it all clicked.
My question for you Alister, do you know how Final Cut Pro X interprets Log? Does it see it as full range, as it should?
FCP-X interprets the internal XAVC recordings correctly.
I’m not entirely sure about externally recorded ProRes however. I think these are handled incorrectly as ProRes is always legal range by default.
Dear Alister, thank you so much for your post.
This Premiere issue was killing me. I had this problem every time I had to work with Alpha A7s footage and I just could not find the right way.
I now fix the data range “problem” in premiere with the technical Lumetri preset “full to legal range, 8-Bit”. That’s even better so you don’t have to set manually to 16 and 235.
Unfortunately the problem consists as soon as I switch to resolve for grading because the xml (most of the time I use the finalcut xml export workflow for timeline transfer between premiere and DaVinci) won’t contain color informations. So in the end the person in charge has to find out which clips must be set to full data levels in the clip attributes (each manually) as soon as the timeline has been transfered to DaVinci. But if you know what to do you you can fix it. Thanks again.
Very interesting, thank you Alister.
This leads me to guess that NLE’s default to Legal when importing footage, unless the files have metadata telling the NLE to assume Data levels. Would that be a correct assumption? And, in that case, any LUTs applied may need to have a Legal>Data conversion applied before the LUT?
I’m also curious about ProRes Raw, and whether it’s Legal or Full range and has metadata.
It does depend on the NLE, but in general, they will default to legal. The main issue is ProRes which is always treated as legal, no matter what the metadata says. ProRes Raw is not a conventional video file, so legal/data range has no meaning or relevance as the file is a data file, not a video file.