[ Main Page || Kodak Contacts || FAQs || File Format || Forums ]

This site is not affiliated with the Eastman Kodak Company.


  
[Register]  [Edit Profile]  [Edit Your Preferences]  [Search]
[Private Messages]  [Memberslist]  [FAQ]  [Login]
Cineon Forum Index Discussions / Questions log encoding Q
Goto page ( 1 | 2 Next Page )
Author log encoding Q
Beak


Joined: Sep 02, 2005
Posts: 10
From: Beak
Posted: 2005-09-02 04:55   
This may sound like a dumb question, but I am an Artist after all(!)

When data is endoded log, are we talking about natural log, or base 10 log to generate the Cineon/DPX values?

TIA

---------Beak
Inferno Artist (Lead)/ Digital Colorist
http://www.beakfx.com
Beak f(x) Inc. - VFX | Design | Color


  View Profile of Beak   Goto the website of Beak      Edit/Delete This Post   Reply with quote
David Hodson


Joined: Nov 27, 2004
Posts: 15
Posted: 2005-09-02 08:40   
Quote:

When data is endoded log, are we talking about natural log, or base 10 log to generate the Cineon/DPX values?



It doesn't really matter, since logs are all the same except for a scale factor, but the calculations are usually given in terms of density, which is the (negative of the) base 10 log of the transmission.


  View Profile of David Hodson   Email David Hodson   Goto the website of David Hodson      Edit/Delete This Post   Reply with quote
Beak


Joined: Sep 02, 2005
Posts: 10
From: Beak
Posted: 2005-09-02 14:42   
Quote:

It doesn't really matter, since logs are all the same except for a scale factor, but the calculations are usually given in terms of density, which is the (negative of the) base 10 log of the transmission.



Thanks for your answer,

?? Hmmmm. I didn't know that. No offence, but are you sure? (*1). Is this true if integers or does the series need to be real numbers? As you might have guessed, logs were not my strong suit until later in life when I became a "Cineon Chump"(*2) in addition to a compositor.
If I could ask you another question: If densities are the negative of log(transmit), which end of the cineon file, which as I understand it is supposed to be digital representative of neg density, is the log compacted part. I was and have been under the impression that the dark areas were the areas where the samples are spread out via log encoding. If this is true, then again the dark areas of a Cineon file are a film _negative_, so wouldn't that mean that the dark areas of a Cineon translate into light areas of a print?
Or is my long help assumption correct in that the dark areas are sampled more than the lights, and that this is somehow taken care of between when it leaves my box and hits the neg, so making a Cinoen more of a digital print than a neg?

I hope this makes sense and that you might find time to answer.


*1:I know you are, this is just a strange way of asking for a brief explination, if you could.
*2: A "Cineon Chump" is the buy who everybody else says knows everything about log. Even if you tell them you don't, they think you are hiding something. This was my case for the last gig I did, though lucky for me my current gig is full of people who are sure that _they_ know everything (!)

Lastly, you might find the RGB->Yxy colorspace viewer/converter an interesting tool. It on my web site, follow the Tools section.


Ciao,
---------Beak
Inferno Artist (Lead)/ Digital Colorist
http://www.beakfx.com
Beak f(x) Inc. - VFX | Design | Color


  View Profile of Beak   Goto the website of Beak      Edit/Delete This Post   Reply with quote
David Hodson


Joined: Nov 27, 2004
Posts: 15
Posted: 2005-09-02 19:27   
Quote:

?? Hmmmm. I didn't know that. No offence, but are you sure?



*chuckle* I try not to comment unless I'm sure. (Although I'm always ready to be corrected - no one's perfect.) I'm not sure what you mean about integers ... the values in the Cineon file are integers, but they represent real values, in a sense.

To take an example (with easy numbers): bright scene -> dense negative -> low transmission (say 1/5). log10 of 1/50 ~= -1.7, negative of that = 1.7, at 0.002 per step gives a pixel value of 850 in the file. Dark scene -> thin negative -> high transmission (say 1/2). log10 of 1/2 ~= -0.3, negative of that = 0.3, at 0.002 per step gives a pixel value of 150 in the file. So brightness gives high values, darkness gives low values, which is handy.

Quote:

I was and have been under the impression that the dark areas were the areas where the samples are spread out via log encoding.



Don't think of it as being "compressed" or "spread out". The eye responds (roughly) to ratios of light intensities, and the log operation converts ratios to equal value steps. Because the relationship between the neg and the print is linear (mostly) in log space, equal steps in the pixel values represent equal brightness ratios in either one.

Hope that all makes sense.


  View Profile of David Hodson   Email David Hodson   Goto the website of David Hodson      Edit/Delete This Post   Reply with quote
Beak


Joined: Sep 02, 2005
Posts: 10
From: Beak
Posted: 2005-09-03 15:21   
Quote:

On 2005-09-02 19:27, David Hodson wrote:

*chuckle* I try not to comment unless I'm sure.


! ha. Hope you read the footnotes. Glad you found that funny, some people take things the wrong way you know.

Quote:

.. the values in the Cineon file are integers, but they represent real values, in a sense.


That was poorly worded on my part, I was trying to ask if the samples are normalized to 1.0. As per your explination, it seems they are. That also explains the (* -1) in the equasion. Since log < 1 is a negative. It all makes more sense now.

Quote:

To take an example (with easy numbers): bright scene -> dense negative -> low transmission (say 1/5). log10 of 1/50 ~= -1.7, negative of that = 1.7,


Uhh, wouldn't 1/5 = 0.2 ? Me thinks that 1/50 was a typo...or I'm really not getting something.

Quote:

at 0.002 per step gives a pixel value of 850 in the file.


Ah, now this is interesting. Where does the 0.002 value come from? Is that what Kodak figured back when? Is that similar to what a densitometer reads off of neg density?


Quote:

So brightness gives high values, darkness gives low values, which is handy.


Handy indeed. Glad to know I was right, now I know why(!) Thanks for explaining just how that works.

Quote:

Don't think of it as being "compressed" or "spread out".


Again, maybe not the best choice of words on my part. Though if you think about it, Kodak figured out how to store ~13 bits of linear data in a 10 bit file. In addition to being the correct luma shift response of humans and film media (a handy coincidence) it is compression of sorts. More like an efficiency I suppose. At least a space saver.

Quote:

The eye responds (roughly) to ratios of light intensities,...[snip].. equal brightness ratios in either one.



Got it. Nice one, mate.

So I thought I'd ask your opinion on an analogy. Not often one gets asked an opinion on an analogy, is it? So I'm cooking (simmering really) an article on the how and why of log space, for a target of compositors and animators (esp. animators, as they are often disconnected from th finishing world). I use money as an example of why things are in log space.

<ahem> "It like amounts that you care about. If you had to put $100,000 into a range of 1-1,000, how would you do it? Well, you would include the small change when it was imporant. Let's say you are 18 years old, and you and your friend are trying to buy beer. You head to the cashier, and it's $4.69 for what you've got. You only have $4.21 between the two of you. The cashier is one of those grump Pakistani dudes who won't let it slide. The $0.48 you are missing is important at that range. Later on in life you are buying a stereo. It's $532.68 on sale. You've got $532 in cash. The guy takes it, as the $0.68 doesn't really matter that much at that price range. Follow this logically until you are buying a $95,000 sports car. Sticker price $95,364.41. You offer $95,000 even. No problem, as $364.41 isn't that big a deal when this mich money is involved.
So when making your scale, you would count the pennies when they mattered, and you'd lose them when they didn't. You'd stop counting tens of dollars after a certain point, too.
This is kind of why log works, it counts the values you need, and starts leaving out ones you don't. If you open up a log->lin 2d LUT in a text editor, you will see that this is indeed the case; as the brightness values get higher, they are counted for less."

What do you think?

Ciao,

------Beak


  View Profile of Beak   Goto the website of Beak      Edit/Delete This Post   Reply with quote
David Hodson


Joined: Nov 27, 2004
Posts: 15
Posted: 2005-09-03 16:59   
Quote:

That was poorly worded on my part, I was trying to ask if the samples are normalized to 1.0. As per your explination, it seems they are.



Well, not really. The pixel values are based on the measured density of the film negative. Density can't be less than 0.0 - that would mean that the film transmits more light than it receives. Theoretically it could get arbitrarily high (by blocking more and more light), but in practice the negative maxes out under 2.0.

Quote:

Uhh, wouldn't 1/5 = 0.2 ? Me thinks that 1/50 was a typo...or I'm really not getting something.



1/5 was a typo. 1/50 = 0.02, log10(0.02) ~= -1.7

Quote:

Ah, now this is interesting. Where does the 0.002 value come from? Is that what Kodak figured back when?



Actually, that was me. We had 10 bits to hold the density value (i.e. the densitometer reading, yes) and it ranged from somewhere above 0.0 to somewhere below 2.0. So I said "divide it by 0.002 and get an integer value in the range 0 to 1023". Not exactly rocket science, but I liked the nice round number.

Quote:

This is kind of why log works, it counts the values you need, and starts leaving out ones you don't. If you open up a log->lin 2d LUT in a text editor, you will see that this is indeed the case; as the brightness values get higher, they are counted for less."



Yup, that's a good analogy.

-- David (Kodak Cineon programmer, 1990-1995)


  View Profile of David Hodson   Email David Hodson   Goto the website of David Hodson      Edit/Delete This Post   Reply with quote
Beak


Joined: Sep 02, 2005
Posts: 10
From: Beak
Posted: 2005-09-05 05:59   
Quote:

Theoretically it could get arbitrarily high (by blocking more and more light), but in practice the negative maxes out under 2.0.


Hmmm, interesting. Another bit in the bucket.


Quote:

1/5 was a typo. 1/50 = 0.02, log10(0.02) ~= -1.7


Add a zero, confuse an artist. Esp. after late night shift on crap horror film to pay mortgage. Shouldn't do math @ 06:30 after work. At least we deliver in a few days.

Quote:

Actually, that was me. We had 10 bits to hold the density value (i.e. the densitometer reading, yes) and it ranged from somewhere above 0.0 to somewhere below 2.0. So I said "divide it by 0.002 and get an integer value in the range 0 to 1023". Not exactly rocket science, but I liked the nice round number.


Now that is some cool stuff! I imageine a scene where several people are discussing things around a monitor, and one guy says (in this case, you) "Hey how about..." and it sticks for ever. I love when that happens. OK, maybe you had to submit a formal Request for Authorization of Concieveable Factors, (form # 101128362/A), but the sitting around scene just works better.

Quote:

Yup, that's a good analogy.



Excelent, I'm glad you like it. It has got me a lot of mileage in bringing people up to speed on why things are happening to them in such strange ways since I showed up. I'm glad to know that the one niggling doubt has been erased, after your first reply.

Quote:

-- David (Kodak Cineon programmer, 1990-1995)




I see. I didn't find your CV anywhere on your site. It is nice to meet you. Say, do you know what happened to the code for Cineon? I heard rumours (like from a friend at GenArts) that the source was for sale, but no follow up. I'm trying to start a spark (Flame/Inferno plug-in) project now/soon. I had targeted Marty Ollstein, who wrote the Tiffen SParks, but he has yet to get back to me. It would be nce to get a hold of some of the log color correction tools. And where is Cinespeed? I thought that was one of the best timewarpers ever. It did take forever to render, though, but that was then. And the grain tools were alway what we called 'cheating' since 'you' had access to all that Kodak R&D, Still kind of the case, if you have ever tried to get a non-encrypted 3d LUT from them.
That reminds me, there is an open3dLUT standard cooking around here (L.A.) from the D-cinema initiative people, This should be a relief to us in the field, who now have to battle Filmlight/discreet/Kodak/Technicolor etc. just to try and get a freaking 3d LUT that we can share. Everybody is in that beginning greed stage right now.

Well, thanks for your time, and the education. Hope to see you around. Email me anytime you like, for a chat or something. I noticed you are in Auz, so no chance for a pint.

Ciao,

-----------------------Beak

[ This Message was edited by: Beak on 2005-09-05 06:00 ]


  View Profile of Beak   Goto the website of Beak      Edit/Delete This Post   Reply with quote
floydboy


Joined: Jul 23, 2002
Posts: 22
Posted: 2005-09-06 03:50   
Ha! They are still pushing that 3D lut stuff? Talk about something that took a while to render. Well, that was back in those days anyways. I wrote that Cineon node back in 1999 for EK when they were just jumping into D-Cinema stuff.

  View Profile of floydboy   Email floydboy   Goto the website of floydboy        Edit/Delete This Post   Reply with quote
David Hodson


Joined: Nov 27, 2004
Posts: 15
Posted: 2005-09-06 05:41   
Quote:

OK, maybe you had to submit a formal Request for Authorization of Concieveable Factors, (form # 101128362/A), but the sitting around scene just works better.



Actually, it was probably me, sitting in front of a monitor with my headphones on, reading and replying to an email. (Real life is so much less interesting than the movies.)

Quote:

I heard rumours (like from a friend at GenArts) that the source was for sale, but no follow up.



I'm not the best person to ask - Cineon was still in good condition when it left Melbourne (heh heh). I believe that some of the technology (probably Cinespeed and the grain stuff, as you say) was sold into other products, but no one was interested in the whole package.



  View Profile of David Hodson   Email David Hodson   Goto the website of David Hodson      Edit/Delete This Post   Reply with quote
floydboy


Joined: Jul 23, 2002
Posts: 22
Posted: 2005-09-07 11:50   
Actually Cineon source code was sold to a company called "Silicon Grail". That company was later bought by Apple and most things have now been incorporated into Shake. I do believe, if memory serves me right, that cinespeed and the grain tools were yanked from it when Apple bought it. It was part of the deal when Silicon Grail bought the Cineon code. The IP from Kodak was not to be passed on if the code was later sold to someone else. I do know that Steve is pissed about it just like when the whole Renderman deal went down.

  View Profile of floydboy   Email floydboy   Goto the website of floydboy        Edit/Delete This Post   Reply with quote
Beak


Joined: Sep 02, 2005
Posts: 10
From: Beak
Posted: 2005-09-08 10:55   
Quote:


Actually, it was probably me, sitting in front of a monitor with my headphones on


Ge that sure is a buzz killer amigo! There goes my miniseries...

I was jus reviewing the thread, and it's daytime. My last shift on this film (The Fog) was last night, we deliver on the 9th, but I have other things I must attend to. The point is, now that I can think a bit, I find a puzzle.
If as you say, Cinoen values are the negative of log(10) transmission * 0.002, and then that (from and earlier post
Quote:

Theoretically it could get arbitrarily high (by blocking more and more light), but in practice the negative maxes out under 2.0.


So what happens with a reading of > 1.0 ? Wouldn't that put you with a negative number? e.g. log(1.2) = ~0.079
negative of that = -0.079
Or is there some form of normalization to unity happening?
Or what did I miss(?)

Thanks again,

----------Beak


  View Profile of Beak   Goto the website of Beak      Edit/Delete This Post   Reply with quote
Beak


Joined: Sep 02, 2005
Posts: 10
From: Beak
Posted: 2005-09-08 11:24   
Quote:

On 2005-09-06 03:50, floydboy wrote:
Ha! They are still pushing that 3D lut stuff? Talk about something that took a while to render.


You sound as you might have a better idea, is ther something in the works we should know about? (And what could it possibly be...)

Quote:

Well, that was back in those days anyways. I wrote that Cineon node back in 1999 for EK when they were just jumping into D-Cinema stuff.


That's cool. I think one of the reasons it's taking so long, is that everyone is holding on to their own version/standard of the 3d LUT. I hope one day soon there will be an open standard that all vendors will populate, or at least "secret" 3d LUT information will drift out into public domain. If it were up to me, I'd embed a 3d LUT inthe header of a DPX file, as an option to the viewer app, or processing app. Something like half of a truelight color cube, Film stock LUT + scanner calibration + xenon lamp data. User would combine that with their own display info, or sw gamma or other crap, and at the end stage, the recorder/projector could read all that directly from the file and adjust as needed. I'm sure DI people wouldn't mind having that info handy. Kind of like a "clip history" if you will.
OK, that is a better idea

Now that you mention the Silicon Grail and Apple thing, I do recall hearing that rumour, too.
This explains why Cinespeed was floating around EK, and prob still is. Didn't know the grain stuff was kept. They're going to have to give it up someday! If they don't someone else will do it for them, measuring and all kind of science fiction to reproduce grain structures of Kodak stock. That would suck for all parties (it'd be like when your wife/girlfriend buys you clothes...something gets projected in the process (no pun intended (sort of)).
I remember now that SG was going to put some of the stuff into Chalice, before it closed.

PS did anyone check out my RGB -> Yxy gadget yet? Just trolling for some feedback, not a pat on the head. I'd like to improve it someday.

Ciao,

------------Beak




[ This Message was edited by: Beak on 2005-09-08 11:33 ]


  View Profile of Beak   Goto the website of Beak      Edit/Delete This Post   Reply with quote
David Hodson


Joined: Nov 27, 2004
Posts: 15
Posted: 2005-09-09 03:38   
Quote:

If as you say, Cinoen values are the negative of log(10) transmission * 0.002, and then that (from and earlier post
Quote:

Theoretically it could get arbitrarily high (by blocking more and more light), but in practice the negative maxes out under 2.0.


So what happens with a reading of > 1.0 ? Wouldn't that put you with a negative number? e.g. log(1.2) = ~0.079
negative of that = -0.079
Or is there some form of normalization to unity happening?



Nope, no normalisation.

Transmission is the fraction of light which passes through the negative. By the laws of physics, that must be between 1.0 (perfectly transparent) and 0.0 (perfectly opaque). A transmission of 1.0 would give a density of -log10(1.0), which = 0.0, for a pixel value of 0.0/0.002 = 0. Transmission of 0.0 would give a density of -log10(0.0), which is infinite ... but luckily negative film doesn't get more opaque than a transmission of about 0.01. -(log10(0.01)) = -(-2.0) = 2.0. Pixel value = 2.0/0.002 = 1000.


  View Profile of David Hodson   Email David Hodson   Goto the website of David Hodson      Edit/Delete This Post   Reply with quote
ahauser


Joined: Sep 16, 2004
Posts: 10
Posted: 2005-09-19 03:58   

Sawatdee Beak,
Its Khun Adrian, Grader from Iloura Thailand....too many years ago to remember no?.
Hey, while we're talking numbers could someone explain how you come up with the results for Offset and Gain?
Im transforming 1D Luts to 3d Luts and although its working I still do not fully understand the concept behind the Gain and Offset formulae.
Could you spell it out for me in plain speak. I look forward to making these transforms free for all access when I get all the input methods and icc color translations sorted.

Adrian


  View Profile of ahauser      Edit/Delete This Post   Reply with quote
Beak


Joined: Sep 02, 2005
Posts: 10
From: Beak
Posted: 2005-09-19 16:02   
Quote:

On 2005-09-19 03:58, ahauser wrote:

Sawatdee Beak,
Its Khun Adrian, Grader from Iloura Thailand....too many years ago to remember no?.
Hey, while we're talking numbers could someone explain how you come up with the results for Offset and Gain?
Im transforming 1D Luts to 3d Luts and although its working I still do not fully understand the concept behind the Gain and Offset formulae.
Could you spell it out for me in plain speak. I look forward to making these transforms free for all access when I get all the input methods and icc color translations sorted.

Adrian


Adrian? Wow, sure has been a while! Funny you should pop up here. Hope you are well.
I'm not sure exactly what you mean when you ask about Offset and Gain. Are you refering to the concept of the terms, or a specific way they are implimented in a module of some sort?
Gain and offset are by them selves easy math, in linear space. Log is a different story, and I suspect this is what you are asking, but not sure.

Can you clearify? What exactly are you trying to do?

**EDIT Hey Adrian, we should take this to a new topic. I think that any responses in this thread might wake up David, i.e. may send email to the thread participants, so to not crowd their mailboxes with perhaps a subject off topic from the original thread topic. Uh, it's late, but I'm sure you see my point.
Can you post your question in a new topic?

Or email me off list, we should catch up after all these years, too.

-----Beak


[ This Message was edited by: Beak on 2005-09-22 01:00 ]


  View Profile of Beak   Goto the website of Beak      Edit/Delete This Post   Reply with quote
Goto page ( 1 | 2 Next Page )
  
Lock this Topic Move this Topic Delete this Topic
Powered by phpBB Version 1.4.2
Copyright © 2000 - 2001 The phpBB Group

phpBB Created this page in 0.235127 seconds.