Feature #8096

introduce Time.current_timestamp

Added by Vipul Amler about 1 year ago. Updated 7 months ago.

[ruby-core:53412]
Status:Feedback
Priority:Low
Assignee:Yukihiro Matsumoto
Category:-
Target version:Ruby 2.1.0

Description

=begin
A lot of scenarios and applications require the use of (({Time.now.toi})) which is used as currenttimestamp in systems.

The introduction of (({Time.currenttimestamp})) {or something with similar with different name} would provide implicit integer timestamp instead of going from
(({Time.now})) -> time
object -> to_i -> integer timestamp value

So instead of

Time.now.to_i # Outputs => 1363274618

one could use

Time.current_timestamp # Outputs => 1363274618

=end


Related issues

Related to ruby-trunk - Feature #8658: Process.clock_gettime Closed 07/19/2013
Related to CommonRuby - Feature #8640: Add Time#elapsed to return nanoseconds since creation Open 07/16/2013

History

#1 Updated by Eric Hodel about 1 year ago

Is one second precision sufficient?

#2 Updated by Vipul Amler about 1 year ago

More like nanosecond. Atleast thats what currently happens, if I am not wrong.

#3 Updated by Charles Nutter about 1 year ago

This is an excellent idea. Being able to get timestamps without creating an entire Time object would be really nice for code that does fine-grained timing.

Here's a benchmark of Time.now.to_i compared to the Java method System.nanoTime (the simplest way on JVM to get a timestamp).

https://gist.github.com/headius/5167793

Calculating -------------------------------------
Time.new.to_i 106740 i/100ms

System.nanoTime 187876 i/100ms

   Time.new.to_i  5457230.7 (±7.3%) i/s -   27111960 in   5.000000s
 System.nanoTime 17952011.5 (±9.2%) i/s -   88301720 in   4.976000s

Given that nanoTime likely makes the same native call this feature would, we could see a similar improvement if such a feature were standard.

May I suggest "Time.timestamp" instead? Do people really want timestamps that aren't "current"?

#4 Updated by Akira Tanaka about 1 year ago

2013/3/15 vipulnsward (Vipul Amler) vipulnsward@gmail.com:

A lot of scenarios and applications require the use of Time.now.toi which is used as currenttimestamp in systems.

I don't recommend the value to send to another host or somewhere which the
way to interpret the value is not known.

If your host don't consider leap seconds but another host consider leap second,
The interpretation of the value (year, month, day, hour, minute, second) will
be different.
--
Tanaka Akira

#5 Updated by Eric Wong about 1 year ago

Tanaka Akira akr@fsij.org wrote:

2013/3/15 vipulnsward (Vipul Amler) vipulnsward@gmail.com:

A lot of scenarios and applications require the use of Time.now.toi which is used as currenttimestamp in systems.

I don't recommend the value to send to another host or somewhere which the
way to interpret the value is not known.

If your host don't consider leap seconds but another host consider leap second,
The interpretation of the value (year, month, day, hour, minute, second) will
be different.

The proposed Time.currenttimestamp is the same as Time.now.toi, but
without allocating the intermediate Time object.

(Returns seconds since Epoch, same as time() function in C/POSIX))

#6 Updated by Nobuyoshi Nakada about 1 year ago

  • Description updated (diff)
  • Status changed from Open to Feedback

vipulnsward (Vipul Amler) wrote:

A lot of scenarios and applications require the use of Time.now.toi which is used as currenttimestamp in systems.

For what purpose is it needed?

#7 Updated by Vipul Amler about 1 year ago

=begin
It could be any number of purposes. On top of my head I could think of remote systems, timestamps persisted to Databases, Event marking, Security Systems, Time conversions, etc (Although these are very bad examples)

Maybe http://en.wikipedia.org/wiki/Timestamping_(computing) or something similar would shed more light on this.
=end

#8 Updated by Motohiro KOSAKI about 1 year ago

(3/15/13 4:36 PM), vipulnsward (Vipul Amler) wrote:

Issue #8096 has been updated by vipulnsward (Vipul Amler).

=begin
It could be any number of purposes. On top of my head I could think of remote systems, timestamps persisted to Databases, Event marking, Security Systems, Time conversions, etc (Although these are very bad examples)

Maybe http://en.wikipedia.org/wiki/Timestamping_(computing) or something similar would shed more light on this.
=end

For this purpose, second is not enough accuracy. Which software want such inaccuracy feature?

#9 Updated by Matthew Kerwin about 1 year ago

MySQL stores/transmits TIMESTAMP fields as integer seconds.

Sent from my phone, so excuse the typos.
On Mar 16, 2013 7:13 AM, "KOSAKI Motohiro" kosaki.motohiro@gmail.com
wrote:

(3/15/13 4:36 PM), vipulnsward (Vipul Amler) wrote:

Issue #8096 has been updated by vipulnsward (Vipul Amler).

=begin
It could be any number of purposes. On top of my head I could think of
remote systems, timestamps persisted to Databases, Event marking, Security
Systems, Time conversions, etc (Although these are very bad examples)

Maybe http://en.wikipedia.org/wiki/Timestamping_(computing) or
something similar would shed more light on this.
=end

For this purpose, second is not enough accuracy. Which software want such
inaccuracy feature?

#11 Updated by Charles Nutter about 1 year ago

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

There's also currentTimeMillis, but I think providing sub-ms precision is a good idea. µs seems sufficient to me.

Definitely would like to see this get into 2.1.

#12 Updated by Yui NARUSE about 1 year ago

headius (Charles Nutter) wrote:

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

There's also currentTimeMillis, but I think providing sub-ms precision is a good idea. µs seems sufficient to me.

Definitely would like to see this get into 2.1.

What class is it?
If Float, it introduces extra error.
If Rational, I feel Time is better.

#13 Updated by Charles Nutter about 1 year ago

naruse (Yui NARUSE) wrote:

What class is it?
If Float, it introduces extra error.
If Rational, I feel Time is better.

It is a Fixnum:

ext-jruby-local ~/projects/jruby $ jruby -e "def Time.timestamp; java.lang.System.nano_time / 1000; end; loop { p Time.timestamp; sleep 1.5 }"
1363742953730794
1363742955254423
1363742956757052
1363742958259167
1363742959762959
1363742961264507
1363742962765680
1363742964267276
1363742965769053
1363742967270540
1363742968773957
1363742970275904
1363742971777002
1363742973279089
...

I think the goal here is to reduce the amount of allocation needed to get a simple integer timestamp.

#14 Updated by Motohiro KOSAKI about 1 year ago

(3/19/13 9:31 PM), headius (Charles Nutter) wrote:

Issue #8096 has been updated by headius (Charles Nutter).

naruse (Yui NARUSE) wrote:

What class is it?
If Float, it introduces extra error.
If Rational, I feel Time is better.

It is a Fixnum:

Fixnum is bad idea because it doesn't have enough bits when using 32bit environment.
Java's System.nanoTime() always return 64bit value.

#15 Updated by Yui NARUSE about 1 year ago

kosaki (Motohiro KOSAKI) wrote:

(3/19/13 9:31 PM), headius (Charles Nutter) wrote:

Issue #8096 has been updated by headius (Charles Nutter).

naruse (Yui NARUSE) wrote:

What class is it?
If Float, it introduces extra error.
If Rational, I feel Time is better.

It is a Fixnum:

Fixnum is bad idea because it doesn't have enough bits when using 32bit environment.
Java's System.nanoTime() always return 64bit value.

At the allocation view, there are 3 types:
1. embeded in VALUE itself
2. embeded in struct RVALUE
3. VALUE and extra data

Fixnum and small Float (flonum) is 1, it doesn't need GC.
Small Time, large Float, small Bignum, small Rational, are 2.

On 32bit, there is no way to embeded microsecond to first types.
So using Bignum is acceptable.

Just a note, the API should use CLOCKMONOTONIC for clockgettime.

#16 Updated by Benoit Daloze about 1 year ago

On 19 March 2013 17:56, headius (Charles Nutter) headius@headius.com wrote:

Issue #8096 has been updated by headius (Charles Nutter).

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

That seems to work on Linux and some others, but what about Windows?
It likely happens to be clock_gettime() or such in UNIX.

The doc says "This method can only be used to measure elapsed time and
is not related to any other notion of system or wall-clock time."

http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime%28%29

There's also currentTimeMillis, but I think providing sub-ms precision is a good idea. µs seems sufficient to me.

Definitely would like to see this get into 2.1.

Feature #8096: introduce Time.current_timestamp
https://bugs.ruby-lang.org/issues/8096#change-37740

Author: vipulnsward (Vipul Amler)
Status: Feedback
Priority: Low
Assignee:
Category:
Target version:

=begin
A lot of scenarios and applications require the use of (({Time.now.toi})) which is used as currenttimestamp in systems.

The introduction of (({Time.currenttimestamp})) {or something with similar with different name} would provide implicit integer timestamp instead of going from
(({Time.now})) -> time
object -> to_i -> integer timestamp value

So instead of

Time.now.to_i # Outputs => 1363274618

one could use

Time.current_timestamp # Outputs => 1363274618

=end

http://bugs.ruby-lang.org/

#17 Updated by Yui NARUSE about 1 year ago

Eregon (Benoit Daloze) wrote:

On 19 March 2013 17:56, headius (Charles Nutter) headius@headius.com wrote:

Issue #8096 has been updated by headius (Charles Nutter).

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

That seems to work on Linux and some others, but what about Windows?
It likely happens to be clock_gettime() or such in UNIX.

Python summarized those time related functions:
http://www.python.org/dev/peps/pep-0418/

The doc says "This method can only be used to measure elapsed time and
is not related to any other notion of system or wall-clock time."

http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime%28%29

Whether wall-clock or monotonic clock is desired is the most important point.

#18 Updated by Eric Wong about 1 year ago

"naruse (Yui NARUSE)" naruse@airemix.jp wrote:

Whether wall-clock or monotonic clock is desired is the most important point.

I would like an option/flag so both are available.

#19 Updated by Motohiro KOSAKI about 1 year ago

(3/20/13 11:14 AM), naruse (Yui NARUSE) wrote:

Issue #8096 has been updated by naruse (Yui NARUSE).

Eregon (Benoit Daloze) wrote:

On 19 March 2013 17:56, headius (Charles Nutter) headius@headius.com wrote:

Issue #8096 has been updated by headius (Charles Nutter).

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

That seems to work on Linux and some others, but what about Windows?
It likely happens to be clock_gettime() or such in UNIX.

Python summarized those time related functions:
http://www.python.org/dev/peps/pep-0418/

The doc says "This method can only be used to measure elapsed time and
is not related to any other notion of system or wall-clock time."

http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime%28%29

Whether wall-clock or monotonic clock is desired is the most important point.

FYI: This is the actual OpenJDK System.nanoTime implementation.
Moreover, JVM know what nanoTime is and it can bypass normal method invocation overhead.


jlong os::javaTimeNanos() {
if (Linux::supportsmonotonicclock()) {
struct timespec tp;
int status = Linux::clockgettime(CLOCKMONOTONIC, &tp);
assert(status == 0, "gettime error");
jlong result = jlong(tp.tvsec) * (1000 * 1000 * 1000) + jlong(tp.tvnsec);
return result;
} else {
timeval time;
int status = gettimeofday(&time, NULL);
assert(status != -1, "linux error");
jlong usecs = jlong(time.tvsec) * (1000 * 1000) + jlong(time.tvusec);
return 1000 * usecs;
}
}


However this is one naive point. almost all programmer think following code makes
always positive.

t0 = System.nanoTime()
...
t1 = System.nanoTime()
t = t1 - t0

however this is not correct when following reasons.

  • some os don't support monotonic.
  • x86 has silly tsc implementation and time run backward when multiple cpus
    even if using monotonic.
    i.e. tsc time depend on individual cpus and os scheduler migrate process
    across cpus transparency. that's intel silly.

    That says, System.nanoTime encourage bad programming. but it works excellent when
    we can ignore disadvantages.

#20 Updated by Charles Nutter about 1 year ago

Responding to some concerns:

  1. nanoTime deficiencies

Yes, System.nanoTime is not monotonic. It is designed to be a timestamp from the start of the process, as fast as possible across platforms, and with as fine-grained resolution as possible. That's one goal of a timestamp. I suppose the primary question here is what the timestamp is wanted for... I can see a few use classes:

  • Runtime metric-gathering (profiling, timing, etc). This case will want to be FAST and FINE grained to the greatest extent possible. Monotonicity may or may not be important, but ensuring low overhead is paramount.
  • Wall clock current time timestamp, for persisting, communicating wall-clock times, possible for runtime metric gathering. This needs to be FAST, but maybe not as fine-grained. Most processes you'd persiste to a database aren't going to need ns or even µs precision. In Java, this would be System.currentTimeMillis, which provides millisecond precision slightly slower than nanoTime.
  • Lower-precision wall clock timestamps, for application-level use. Again, you'd want this to be fast, but at this level ms or even second precision may be acceptable.

The same code I posted could be monotonic with System.currentTimeMillis; it just wouldn't be able to provide µs or ns precision.

  1. 32-bit Fixnum is not big enough

I sometimes forget that MRI isn't always using 64-bit numeric values. In JRuby, Fixnum and Float are always 64-bit, regardless of the JVM or platform you're running on. As kosaki pointed out, however, a small Bignum (e.g. 64-bit precision) would still be much smaller than a full Time object, so I think returning Bignum on 32-bit platforms is quite acceptable. JRuby would likely always just return Fixnum.

  1. Other issues with timestamping

There's various gotchas people run into when using even the JVM's fastest timestamp facilities. Among them:

  • nanoTime does not reflect time spent sleeping; if you suspend the machine, the CPU clocks don't advance, so nanoTime does not advance.
  • Accurate clocks can be expensive to provide in virtualized environments. I have confirmed cases where System.currentTimeMillis performed terribly under certain VMs that tried to guarantee accuracy with regards to the host system as well as the virtualized runtime. This is more a kernel-level thing though; if you're using a kernel-level API that claims to be monotonic, there's a good chance some VM software will bend over backwards to support that guarantee.

In any case, I think we can all agree that having to construct an entire Time object every time you want a timestamp is unreasonable...especially since Time gets its value from a kernel-level call we could simply expose to the user directly.

#21 Updated by Vipul Amler about 1 year ago

After the conversation, I hope its reasonable implement this.

#22 Updated by Charles Nutter about 1 year ago

vipulnsward (Vipul Amler) wrote:

After the conversation, I hope its reasonable implement this.

Indeed!

Any other input here? It seems like a proof-of-concept patch for MRI would be a next step. Obviously I can also come up with one in seconds for JRuby...if that's interesting to anyone.

#23 Updated by Matthew Kerwin about 1 year ago

On Apr 13, 2013 2:33 AM, "headius (Charles Nutter)" headius@headius.com
wrote:

Any other input here? It seems like a proof-of-concept patch for MRI
would be a next step. Obviously I can also come up with one in seconds for
JRuby...if that's interesting to anyone.

There is this: https://github.com/phluid61/timestamp-gem

The java stuff is experimentation on my part, published version of the gem
has none of that.

#24 Updated by Charles Nutter about 1 year ago

phluid61 (Matthew Kerwin) wrote:

There is this: https://github.com/phluid61/timestamp-gem

The java stuff is experimentation on my part, published version of the gem
has none of that.

You're pretty close. System.nanoTime is probably the closes thing to your "timestamp", but as kosaki points out it's not guaranteed to be monotonic (even your impl is not monotonic).

Why? Because on some Linux or BSD, monotonic time may not be supported. Time is then per-CPU, which means it can appear to go backward if read from one CPU and then read from another CPU. On Solaris, the old gethrtime has the same problem, but Java attempts to do a monotonic version with some processor tricks.

Here's the Windows impl from OpenJDK:

jlong os::javaTimeNanos() {
if (!hasperformancecount) {
return javaTimeMillis() * NANOSECSPERMILLISEC; // the best we can do.
} else {
LARGEINTEGER currentcount;
QueryPerformanceCounter(&currentcount);
double current = as
long(currentcount);
double freq = performance
frequency;
jlong time = (jlong)((current/freq) * NANOSECSPERSEC);
return time;
}
}

Where javaTimeMillis is:

jlong os::javaTimeMillis() {
if (UseFakeTimers) {
return faketime++;
} else {
FILETIME wt;
GetSystemTimeAsFileTime(&wt);
return windows
tojavatime(wt);
}
}

Obviously getting "true time" is much more complicated than just finding the right call.

Your implementation for MRI is probably just fine. I can help make the JRuby version "correct" using nanoTime and proper method binding. But a few questions remain:

  • Do we want a single Time.timestamp method?
  • If so, which timestamp is it?
  • If not, how many representations of timestamp do we want? It seems to me that our timestamp and Time.now.to_i should probably be derived the same way.

#25 Updated by Charles Nutter about 1 year ago

JRuby patch to add Time.timestamp/current_timestamp and Time.microtime: https://gist.github.com/headius/5393552

This is based on system time (System.currentTimeMillis), not CPU time (System.nanoTime) as in the 'jruby' branch of pluid61's timestamp-gem.

#26 Updated by Matthew Kerwin about 1 year ago

headius (Charles Nutter) wrote:

phluid61 (Matthew Kerwin) wrote:

There is this: https://github.com/phluid61/timestamp-gem

The java stuff is experimentation on my part, published version of the gem
has none of that.

You're pretty close. System.nanoTime is probably the closes thing to your "timestamp", but as kosaki points out it's not guaranteed to be monotonic (even your impl is not monotonic).

Actually since making the jruby branch, and based on some conversations here, I have changed the API of the gem (and the C implementation in master) to define a monotonic Time.timestamp method and a Time.now.toi-equivalent wall-clock method, which I called Time.unixtimestamp because I couldn't think of a better name.

My intention was to update the jruby branch to use System.nanoTime for Time.timestamp, which I've finally gotten around to doing, but I see there are still potentially some polytonic (is that even a word?) issues to resolve. Fortunately I've not yet set up my VM to build jruby/java-native gems, or verified my hopefully MRI-and-jruby compatible gemspec, so my java code won't be released into the wild for some time yet.

Your implementation for MRI is probably just fine. I can help make the JRuby version "correct" using nanoTime and proper method binding. But a few questions remain:

  • Do we want a single Time.timestamp method?
  • If so, which timestamp is it?
  • If not, how many representations of timestamp do we want? It seems to me that our timestamp and Time.now.to_i should probably be derived the same way.

This discussion has already revealed, to me at least, that some people want a fast seconds-since-1969 method, and some people want a secure, high-precision monotonicically increasing value. I think the latter has the most merit, since there is currently no equivalent in Ruby; however both are legitimate requests.

#27 Updated by Charles Nutter about 1 year ago

phluid61 (Matthew Kerwin) wrote:

Actually since making the jruby branch, and based on some conversations here, I have changed the API of the gem (and the C implementation in master) to define a monotonic Time.timestamp method and a Time.now.toi-equivalent wall-clock method, which I called Time.unixtimestamp because I couldn't think of a better name.

My intention was to update the jruby branch to use System.nanoTime for Time.timestamp, which I've finally gotten around to doing, but I see there are still potentially some polytonic (is that even a word?) issues to resolve. Fortunately I've not yet set up my VM to build jruby/java-native gems, or verified my hopefully MRI-and-jruby compatible gemspec, so my java code won't be released into the wild for some time yet.

I could submit a PR for master that provides the same methods in a JRuby-compatible way, if you'd like.

  • Do we want a single Time.timestamp method?
  • If so, which timestamp is it?
  • If not, how many representations of timestamp do we want? It seems to me that our timestamp and Time.now.to_i should probably be derived the same way.

This discussion has already revealed, to me at least, that some people want a fast seconds-since-1969 method, and some people want a secure, high-precision monotonicically increasing value. I think the latter has the most merit, since there is currently no equivalent in Ruby; however both are legitimate requests.

Yes, those seem like the key points. The monotonicity guarantees of the latter are hard to meet, though...on any impl. it might have to be specified as "high-precision CPU-level timestamp, which will be monotonic on most systems (but this is not guaranteed)".

So we have:

  • Time.timestamp: A ns-precision timestamp based on the fastest, most accurate timer possible on the current platform. Although not guaranteed, this value will be monotonic on most systems.
  • Time.something_timestamp: A timestamp representing the number of seconds since the epoch.

I suggest epoch_timestamp for the latter.

Would this meet everyone's needs?

#28 Updated by Matthew Kerwin about 1 year ago

headius (Charles Nutter) wrote:

phluid61 (Matthew Kerwin) wrote:

My intention was to update the jruby branch to use System.nanoTime for Time.timestamp, which I've finally gotten around to doing, but I see there are still potentially some polytonic (is that even a word?) issues to resolve. Fortunately I've not yet set up my VM to build jruby/java-native gems, or verified my hopefully MRI-and-jruby compatible gemspec, so my java code won't be released into the wild for some time yet.

I could submit a PR for master that provides the same methods in a JRuby-compatible way, if you'd like.

That would be really handy. I've been fiddling a bit more the past couple of days, and I can't quite get it to work properly. I don't know if the failure is in the class file, the jar, or loading it in the test.rb

So we have:

  • Time.timestamp: A ns-precision timestamp based on the fastest, most accurate timer possible on the current platform. Although not guaranteed, this value will be monotonic on most systems.
  • Time.something_timestamp: A timestamp representing the number of seconds since the epoch.

I suggest epoch_timestamp for the latter.

I chose unixtimestamp because that's what MySQL calls it (UNIXTIMESTAMP and FROMUNIXTIME), and a quick and very scientific Google poll shows me that "unix timestamp" has ~893k results, "epoch timestamp" ~463k, "posix timestamp" ~430k; and the second, third, and fifth results for "timestamp" are www.unixtimestamp.com/, www.epochconverter.com/ (the "Easy epoch/Unix timestamp converter for computer programmers.") and www.onlineconversion.com/unixtime.htm, etc. so the "Unix" name is in pretty common usage.

#29 Updated by Charles Nutter 7 months ago

  • Assignee set to Akira Tanaka
  • Target version set to Ruby 2.1.0

This is now provided by #8658 in 2.1, but according to matz in the developer meeting notes linked there, Process::clock_gettime is not considered "spec".

So I'm not sure if this can be closed or not...we need to decide.

Optimistically assigning to akr.

#30 Updated by Akira Tanaka 7 months ago

  • Assignee deleted (Akira Tanaka)

I can not decide Ruby's specification.

#31 Updated by Yui NARUSE 7 months ago

  • Assignee set to Yukihiro Matsumoto

headius (Charles Nutter) wrote:

This is now provided by #8658 in 2.1, but according to matz in the developer meeting notes linked there, Process::clock_gettime is not considered "spec".

So I'm not sure if this can be closed or not...we need to decide.

Optimistically assigning to akr.

Process.clock_gettime is low level API; it is different from this high level API.
On the meeting, matz considered this high level API didn't get consensus yet.

#32 Updated by Charles Nutter 7 months ago

It would not be hard, given the guarantee that CLOCKREALTIME works across platforms, to provide a #timestamp that is always CLOCKREALTIME.

Can we do that, so we have an official API?

#33 Updated by Yui NARUSE 7 months ago

headius (Charles Nutter) wrote:

It would not be hard, given the guarantee that CLOCKREALTIME works across platforms, to provide a #timestamp that is always CLOCKREALTIME.

Can we do that, so we have an official API?

We can but whether is is official or not is matz's decision.
Moreover Process.clock_gettime is different issue from this ticket.

Also available in: Atom PDF