Posts: 90
Threads: 4
Joined: Aug 2009
Reputation:
0
10-12-2011, 10:08 PM
(This post was last modified: 10-12-2011, 10:10 PM by taltamir.)
(10-12-2011, 09:58 PM)avih Wrote: But it is truncated. The float type has 24 bits precision, or about 7 decimal digits. Anything longer is truncated.
Ah, i see what you mean now. 59.94005994005994005994005994006 isn't truncated. But if we try to input it into "float" (the command) it will get truncated because a float is a 24bit floating decimal number which can only store so many significant digits.
What about using a double then? it should be good for 52 significand and the number above is only 31.
I do not have a superman complex; for I am God, not superman!
Rig: Q9400, 4GB DDR2, eVGA GTX260 SC, gigabyte EP35-DS3R. X25-M 80GB G2.
Posts: 90
Threads: 4
Joined: Aug 2009
Reputation:
0
(10-12-2011, 10:18 PM)avih Wrote: And it is good enough. Recall it was 60 before these changes, then eventually changed from 2 digits precision to 7 digits, making it 100K times more accurate than it originally was. The implications of the original inaccuracy was 1 second A/V drift every 15 minutes. 59.94 would have been 1s every 1500 minutes, and now it's 1 second every 1.5 million minutes of recording.
My understanding was that the current value is 59.94
What is the current value that has an inaccuracy of 1 second every 1.5 million minutes of recording?
I do not have a superman complex; for I am God, not superman!
Rig: Q9400, 4GB DDR2, eVGA GTX260 SC, gigabyte EP35-DS3R. X25-M 80GB G2.
Posts: 90
Threads: 4
Joined: Aug 2009
Reputation:
0
(10-13-2011, 12:39 AM)avih Wrote: Current value is (more of less) the best approximation at which a float can represent the result of (30/1.001)*2, which happens to be 59.940056
I see, thank you for clarifying.
I do not have a superman complex; for I am God, not superman!
Rig: Q9400, 4GB DDR2, eVGA GTX260 SC, gigabyte EP35-DS3R. X25-M 80GB G2.