Feature #8096: introduce Time.current_timestamp

• Status: Feedback
• Assignee: matz (Yukihiro Matsumoto)
• Target version: -

[vipulnsward (Vipul Amler)]

=begin
A lot of scenarios and applications require the use of (({Time.now.to_i})) which is used as current_timestamp in systems.

The introduction of (({Time.current_timestamp})) {or something with similar with different name} would provide implicit integer timestamp instead of going from
(({Time.now})) -> time_object -> to_i -> integer timestamp value

So instead of

Time.now.to_i # Outputs => 1363274618

one could use

Time.current_timestamp # Outputs => 1363274618

=end

[drbrain (Eric Hodel)]

Is one second precision sufficient?

[vipulnsward (Vipul Amler)]

More like nanosecond. Atleast thats what currently happens, if I am not wrong.

[headius (Charles Nutter)]

This is an excellent idea. Being able to get timestamps without creating an entire Time object would be really nice for code that does fine-grained timing.

Here's a benchmark of Time.now.to_i compared to the Java method System.nanoTime (the simplest way on JVM to get a timestamp).

https://gist.github.com/headius/5167793

Calculating -------------------------------------
Time.new.to_i 106740 i/100ms
System.nanoTime 187876 i/100ms

   Time.new.to_i  5457230.7 (±7.3%) i/s -   27111960 in   5.000000s
 System.nanoTime 17952011.5 (±9.2%) i/s -   88301720 in   4.976000s

Given that nanoTime likely makes the same native call this feature would, we could see a similar improvement if such a feature were standard.

May I suggest "Time.timestamp" instead? Do people really want timestamps that aren't "current"?

[akr (Akira Tanaka)]

2013/3/15 vipulnsward (Vipul Amler) [e-mail]:

A lot of scenarios and applications require the use of Time.now.to_i which is used as current_timestamp in systems.

I don't recommend the value to send to another host or somewhere which the
way to interpret the value is not known.

If your host don't consider leap seconds but another host consider leap second,
The interpretation of the value (year, month, day, hour, minute, second) will
be different.

Tanaka Akira

[normalperson (Eric Wong)]

Tanaka Akira [e-mail] wrote:

2013/3/15 vipulnsward (Vipul Amler) [e-mail]:

A lot of scenarios and applications require the use of Time.now.to_i which is used as current_timestamp in systems.

I don't recommend the value to send to another host or somewhere which the
way to interpret the value is not known.

If your host don't consider leap seconds but another host consider leap second,
The interpretation of the value (year, month, day, hour, minute, second) will
be different.

The proposed Time.current_timestamp is the same as Time.now.to_i, but
without allocating the intermediate Time object.

(Returns seconds since Epoch, same as time() function in C/POSIX))

[nobu (Nobuyoshi Nakada)]

• Description updated (diff)
• Status changed from Open to Feedback

vipulnsward (Vipul Amler) wrote:

A lot of scenarios and applications require the use of Time.now.to_i which is used as current_timestamp in systems.

For what purpose is it needed?

[vipulnsward (Vipul Amler)]

=begin
It could be any number of purposes. On top of my head I could think of remote systems, timestamps persisted to Databases, Event marking, Security Systems, Time conversions, etc (Although these are very bad examples)

Maybe http://en.wikipedia.org/wiki/Timestamping_(computing) or something similar would shed more light on this.
=end

[kosaki (Motohiro KOSAKI)]

(3/15/13 4:36 PM), vipulnsward (Vipul Amler) wrote:

Issue #8096 has been updated by vipulnsward (Vipul Amler).

=begin
It could be any number of purposes. On top of my head I could think of remote systems, timestamps persisted to Databases, Event marking, Security Systems, Time conversions, etc (Although these are very bad examples)

Maybe http://en.wikipedia.org/wiki/Timestamping_(computing) or something similar would shed more light on this.
=end

For this purpose, second is not enough accuracy. Which software want such inaccuracy feature?

[phluid61 (Matthew Kerwin)]

MySQL stores/transmits TIMESTAMP fields as integer seconds.

Sent from my phone, so excuse the typos.
On Mar 16, 2013 7:13 AM, "KOSAKI Motohiro" [e-mail]
wrote:

(3/15/13 4:36 PM), vipulnsward (Vipul Amler) wrote:

Issue #8096 has been updated by vipulnsward (Vipul Amler).

=begin
It could be any number of purposes. On top of my head I could think of
remote systems, timestamps persisted to Databases, Event marking, Security
Systems, Time conversions, etc (Although these are very bad examples)

Maybe http://en.wikipedia.org/wiki/Timestamping_(computing) or
something similar would shed more light on this.
=end

For this purpose, second is not enough accuracy. Which software want such
inaccuracy feature?

[drbrain (Eric Hodel)]

Both Postgres and MySQL store timestamps with microsecond precision:

http://www.postgresql.org/docs/9.2/static/datatype-datetime.html

http://dev.mysql.com/doc/refman/5.6/en/datetime.html

[headius (Charles Nutter)]

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

There's also currentTimeMillis, but I think providing sub-ms precision is a good idea. µs seems sufficient to me.

Definitely would like to see this get into 2.1.

[naruse (Yui NARUSE)]

headius (Charles Nutter) wrote:

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

There's also currentTimeMillis, but I think providing sub-ms precision is a good idea. µs seems sufficient to me.

Definitely would like to see this get into 2.1.

What class is it?
If Float, it introduces extra error.
If Rational, I feel Time is better.

[headius (Charles Nutter)]

naruse (Yui NARUSE) wrote:

What class is it?
If Float, it introduces extra error.
If Rational, I feel Time is better.

It is a Fixnum:

ext-jruby-local ~/projects/jruby $ jruby -e "def Time.timestamp; java.lang.System.nano_time / 1000; end; loop { p Time.timestamp; sleep 1.5 }"
1363742953730794
1363742955254423
1363742956757052
1363742958259167
1363742959762959
1363742961264507
1363742962765680
1363742964267276
1363742965769053
1363742967270540
1363742968773957
1363742970275904
1363742971777002
1363742973279089
...

I think the goal here is to reduce the amount of allocation needed to get a simple integer timestamp.

[kosaki (Motohiro KOSAKI)]

(3/19/13 9:31 PM), headius (Charles Nutter) wrote:

Issue #8096 has been updated by headius (Charles Nutter).

naruse (Yui NARUSE) wrote:

What class is it?
If Float, it introduces extra error.
If Rational, I feel Time is better.

It is a Fixnum:

Fixnum is bad idea because it doesn't have enough bits when using 32bit environment.
Java's System.nanoTime() always return 64bit value.

[naruse (Yui NARUSE)]

kosaki (Motohiro KOSAKI) wrote:

(3/19/13 9:31 PM), headius (Charles Nutter) wrote:

Issue #8096 has been updated by headius (Charles Nutter).

naruse (Yui NARUSE) wrote:

What class is it?
If Float, it introduces extra error.
If Rational, I feel Time is better.

It is a Fixnum:

Fixnum is bad idea because it doesn't have enough bits when using 32bit environment.
Java's System.nanoTime() always return 64bit value.

At the allocation view, there are 3 types:

1. embeded in VALUE itself
2. embeded in struct RVALUE
3. VALUE and extra data

Fixnum and small Float (flonum) is 1, it doesn't need GC.
Small Time, large Float, small Bignum, small Rational, are 2.

On 32bit, there is no way to embeded microsecond to first types.
So using Bignum is acceptable.

Just a note, the API should use CLOCK_MONOTONIC for clock_gettime.

[Eregon (Benoit Daloze)]

On 19 March 2013 17:56, headius (Charles Nutter) [e-mail] wrote:

Issue #8096 has been updated by headius (Charles Nutter).

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

That seems to work on Linux and some others, but what about Windows?
It likely happens to be clock_gettime() or such in UNIX.

The doc says "This method can only be used to measure elapsed time and
is not related to any other notion of system or wall-clock time."

http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime%28%29

There's also currentTimeMillis, but I think providing sub-ms precision is a good idea. µs seems sufficient to me.

Definitely would like to see this get into 2.1.

Feature #8096: introduce Time.current_timestamp
https://bugs.ruby-lang.org/issues/8096#change-37740

Author: vipulnsward (Vipul Amler)
Status: Feedback
Priority: Low
Assignee:
Category:
Target version:

=begin
A lot of scenarios and applications require the use of (({Time.now.to_i})) which is used as current_timestamp in systems.

The introduction of (({Time.current_timestamp})) {or something with similar with different name} would provide implicit integer timestamp instead of going from
(({Time.now})) -> time_object -> to_i -> integer timestamp value

So instead of

Time.now.to_i # Outputs => 1363274618

one could use

Time.current_timestamp # Outputs => 1363274618

=end

--
http://bugs.ruby-lang.org/

[naruse (Yui NARUSE)]

Eregon (Benoit Daloze) wrote:

On 19 March 2013 17:56, headius (Charles Nutter) [e-mail] wrote:

Issue #8096 has been updated by headius (Charles Nutter).

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

That seems to work on Linux and some others, but what about Windows?
It likely happens to be clock_gettime() or such in UNIX.

Python summarized those time related functions:
http://www.python.org/dev/peps/pep-0418/

The doc says "This method can only be used to measure elapsed time and
is not related to any other notion of system or wall-clock time."

http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime%28%29

Whether wall-clock or monotonic clock is desired is the most important point.

[normalperson (Eric Wong)]

"naruse (Yui NARUSE)" [e-mail] wrote:

Whether wall-clock or monotonic clock is desired is the most important point.

I would like an option/flag so both are available.

[kosaki (Motohiro KOSAKI)]

(3/20/13 11:14 AM), naruse (Yui NARUSE) wrote:

Issue #8096 has been updated by naruse (Yui NARUSE).

Eregon (Benoit Daloze) wrote:

On 19 March 2013 17:56, headius (Charles Nutter) [e-mail] wrote:

Issue #8096 has been updated by headius (Charles Nutter).

A few more thoughts to keep this alive...

Implementation in JRuby:

def Time.timestamp
java.lang.System.nano_time / 1000
end

That seems to work on Linux and some others, but what about Windows?
It likely happens to be clock_gettime() or such in UNIX.

Python summarized those time related functions:
http://www.python.org/dev/peps/pep-0418/

The doc says "This method can only be used to measure elapsed time and
is not related to any other notion of system or wall-clock time."

http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime%28%29

Whether wall-clock or monotonic clock is desired is the most important point.

FYI: This is the actual OpenJDK System.nanoTime implementation.
Moreover, JVM know what nanoTime is and it can bypass normal method invocation overhead.

---

jlong os::javaTimeNanos() {
if (Linux::supports_monotonic_clock()) {
struct timespec tp;
int status = Linux::clock_gettime(CLOCK_MONOTONIC, &tp);
assert(status == 0, "gettime error");
jlong result = jlong(tp.tv_sec) * (1000 * 1000 * 1000) + jlong(tp.tv_nsec);
return result;
} else {
timeval time;
int status = gettimeofday(&time, NULL);
assert(status != -1, "linux error");
jlong usecs = jlong(time.tv_sec) * (1000 * 1000) + jlong(time.tv_usec);
return 1000 * usecs;
}
}

However this is one naive point. almost all programmer think following code makes
always positive.

t0 = System.nanoTime()
...
t1 = System.nanoTime()
t = t1 - t0

however this is not correct when following reasons.

• some os don't support monotonic.
• x86 has silly tsc implementation and time run backward when multiple cpus
  even if using monotonic.
  i.e. tsc time depend on individual cpus and os scheduler migrate process
  across cpus transparency. that's intel silly.

That says, System.nanoTime encourage bad programming. but it works excellent when
we can ignore disadvantages.

[headius (Charles Nutter)]

Responding to some concerns:

1. nanoTime deficiencies

Yes, System.nanoTime is not monotonic. It is designed to be a timestamp from the start of the process, as fast as possible across platforms, and with as fine-grained resolution as possible. That's one goal of a timestamp. I suppose the primary question here is what the timestamp is wanted for... I can see a few use classes:

• Runtime metric-gathering (profiling, timing, etc). This case will want to be FAST and FINE grained to the greatest extent possible. Monotonicity may or may not be important, but ensuring low overhead is paramount.
• Wall clock current time timestamp, for persisting, communicating wall-clock times, possible for runtime metric gathering. This needs to be FAST, but maybe not as fine-grained. Most processes you'd persiste to a database aren't going to need ns or even µs precision. In Java, this would be System.currentTimeMillis, which provides millisecond precision slightly slower than nanoTime.
• Lower-precision wall clock timestamps, for application-level use. Again, you'd want this to be fast, but at this level ms or even second precision may be acceptable.

The same code I posted could be monotonic with System.currentTimeMillis; it just wouldn't be able to provide µs or ns precision.

2. 32-bit Fixnum is not big enough

I sometimes forget that MRI isn't always using 64-bit numeric values. In JRuby, Fixnum and Float are always 64-bit, regardless of the JVM or platform you're running on. As kosaki pointed out, however, a small Bignum (e.g. 64-bit precision) would still be much smaller than a full Time object, so I think returning Bignum on 32-bit platforms is quite acceptable. JRuby would likely always just return Fixnum.

3. Other issues with timestamping

There's various gotchas people run into when using even the JVM's fastest timestamp facilities. Among them:

• nanoTime does not reflect time spent sleeping; if you suspend the machine, the CPU clocks don't advance, so nanoTime does not advance.
• Accurate clocks can be expensive to provide in virtualized environments. I have confirmed cases where System.currentTimeMillis performed terribly under certain VMs that tried to guarantee accuracy with regards to the host system as well as the virtualized runtime. This is more a kernel-level thing though; if you're using a kernel-level API that claims to be monotonic, there's a good chance some VM software will bend over backwards to support that guarantee.

In any case, I think we can all agree that having to construct an entire Time object every time you want a timestamp is unreasonable...especially since Time gets its value from a kernel-level call we could simply expose to the user directly.

[vipulnsward (Vipul Amler)]

After the conversation, I hope its reasonable implement this.

[headius (Charles Nutter)]

vipulnsward (Vipul Amler) wrote:

After the conversation, I hope its reasonable implement this.

Indeed!

Any other input here? It seems like a proof-of-concept patch for MRI would be a next step. Obviously I can also come up with one in seconds for JRuby...if that's interesting to anyone.

[phluid61 (Matthew Kerwin)]

On Apr 13, 2013 2:33 AM, "headius (Charles Nutter)" [e-mail]
wrote:

Any other input here? It seems like a proof-of-concept patch for MRI
would be a next step. Obviously I can also come up with one in seconds for
JRuby...if that's interesting to anyone.

There is this: https://github.com/phluid61/timestamp-gem

The java stuff is experimentation on my part, published version of the gem
has none of that.

[headius (Charles Nutter)]

phluid61 (Matthew Kerwin) wrote:

There is this: https://github.com/phluid61/timestamp-gem

The java stuff is experimentation on my part, published version of the gem
has none of that.

You're pretty close. System.nanoTime is probably the closes thing to your "timestamp", but as kosaki points out it's not guaranteed to be monotonic (even your impl is not monotonic).

Why? Because on some Linux or BSD, monotonic time may not be supported. Time is then per-CPU, which means it can appear to go backward if read from one CPU and then read from another CPU. On Solaris, the old gethrtime has the same problem, but Java attempts to do a monotonic version with some processor tricks.

Here's the Windows impl from OpenJDK:

jlong os::javaTimeNanos() {
if (!has_performance_count) {
return javaTimeMillis() * NANOSECS_PER_MILLISEC; // the best we can do.
} else {
LARGE_INTEGER current_count;
QueryPerformanceCounter(&current_count);
double current = as_long(current_count);
double freq = performance_frequency;
jlong time = (jlong)((current/freq) * NANOSECS_PER_SEC);
return time;
}
}

Where javaTimeMillis is:

jlong os::javaTimeMillis() {
if (UseFakeTimers) {
return fake_time++;
} else {
FILETIME wt;
GetSystemTimeAsFileTime(&wt);
return windows_to_java_time(wt);
}
}

Obviously getting "true time" is much more complicated than just finding the right call.

Your implementation for MRI is probably just fine. I can help make the JRuby version "correct" using nanoTime and proper method binding. But a few questions remain:

• Do we want a single Time.timestamp method?
• If so, which timestamp is it?
• If not, how many representations of timestamp do we want? It seems to me that our timestamp and Time.now.to_i should probably be derived the same way.

[headius (Charles Nutter)]

JRuby patch to add Time.timestamp/current_timestamp and Time.microtime: https://gist.github.com/headius/5393552

This is based on system time (System.currentTimeMillis), not CPU time (System.nanoTime) as in the 'jruby' branch of pluid61's timestamp-gem.

[phluid61 (Matthew Kerwin)]

headius (Charles Nutter) wrote:

phluid61 (Matthew Kerwin) wrote:

There is this: https://github.com/phluid61/timestamp-gem

The java stuff is experimentation on my part, published version of the gem
has none of that.

You're pretty close. System.nanoTime is probably the closes thing to your "timestamp", but as kosaki points out it's not guaranteed to be monotonic (even your impl is not monotonic).

Actually since making the jruby branch, and based on some conversations here, I have changed the API of the gem (and the C implementation in master) to define a monotonic Time.timestamp method and a Time.now.to_i-equivalent wall-clock method, which I called Time.unix_timestamp because I couldn't think of a better name.

My intention was to update the jruby branch to use System.nanoTime for Time.timestamp, which I've finally gotten around to doing, but I see there are still potentially some polytonic (is that even a word?) issues to resolve. Fortunately I've not yet set up my VM to build jruby/java-native gems, or verified my hopefully MRI-and-jruby compatible gemspec, so my java code won't be released into the wild for some time yet.

Your implementation for MRI is probably just fine. I can help make the JRuby version "correct" using nanoTime and proper method binding. But a few questions remain:

• Do we want a single Time.timestamp method?
• If so, which timestamp is it?
• If not, how many representations of timestamp do we want? It seems to me that our timestamp and Time.now.to_i should probably be derived the same way.

This discussion has already revealed, to me at least, that some people want a fast seconds-since-1969 method, and some people want a secure, high-precision monotonicically increasing value. I think the latter has the most merit, since there is currently no equivalent in Ruby; however both are legitimate requests.

[headius (Charles Nutter)]

phluid61 (Matthew Kerwin) wrote:

Actually since making the jruby branch, and based on some conversations here, I have changed the API of the gem (and the C implementation in master) to define a monotonic Time.timestamp method and a Time.now.to_i-equivalent wall-clock method, which I called Time.unix_timestamp because I couldn't think of a better name.

My intention was to update the jruby branch to use System.nanoTime for Time.timestamp, which I've finally gotten around to doing, but I see there are still potentially some polytonic (is that even a word?) issues to resolve. Fortunately I've not yet set up my VM to build jruby/java-native gems, or verified my hopefully MRI-and-jruby compatible gemspec, so my java code won't be released into the wild for some time yet.

I could submit a PR for master that provides the same methods in a JRuby-compatible way, if you'd like.

• Do we want a single Time.timestamp method?
• If so, which timestamp is it?
• If not, how many representations of timestamp do we want? It seems to me that our timestamp and Time.now.to_i should probably be derived the same way.

This discussion has already revealed, to me at least, that some people want a fast seconds-since-1969 method, and some people want a secure, high-precision monotonicically increasing value. I think the latter has the most merit, since there is currently no equivalent in Ruby; however both are legitimate requests.

Yes, those seem like the key points. The monotonicity guarantees of the latter are hard to meet, though...on any impl. it might have to be specified as "high-precision CPU-level timestamp, which will be monotonic on most systems (but this is not guaranteed)".

So we have:

• Time.timestamp: A ns-precision timestamp based on the fastest, most accurate timer possible on the current platform. Although not guaranteed, this value will be monotonic on most systems.
• Time.something_timestamp: A timestamp representing the number of seconds since the epoch.

I suggest epoch_timestamp for the latter.

Would this meet everyone's needs?

[phluid61 (Matthew Kerwin)]

headius (Charles Nutter) wrote:

phluid61 (Matthew Kerwin) wrote:

My intention was to update the jruby branch to use System.nanoTime for Time.timestamp, which I've finally gotten around to doing, but I see there are still potentially some polytonic (is that even a word?) issues to resolve. Fortunately I've not yet set up my VM to build jruby/java-native gems, or verified my hopefully MRI-and-jruby compatible gemspec, so my java code won't be released into the wild for some time yet.

I could submit a PR for master that provides the same methods in a JRuby-compatible way, if you'd like.

That would be really handy. I've been fiddling a bit more the past couple of days, and I can't quite get it to work properly. I don't know if the failure is in the class file, the jar, or loading it in the test.rb

So we have:

• Time.timestamp: A ns-precision timestamp based on the fastest, most accurate timer possible on the current platform. Although not guaranteed, this value will be monotonic on most systems.
• Time.something_timestamp: A timestamp representing the number of seconds since the epoch.

I suggest epoch_timestamp for the latter.

I chose unix_timestamp because that's what MySQL calls it (UNIX_TIMESTAMP and FROM_UNIXTIME), and a quick and very scientific Google poll shows me that "unix timestamp" has ~893k results, "epoch timestamp" ~463k, "posix timestamp" ~430k; and the second, third, and fifth results for "timestamp" are www.unixtimestamp.com/, www.epochconverter.com/ (the "Easy epoch/Unix timestamp converter for computer programmers.") and www.onlineconversion.com/unix_time.htm, etc. so the "Unix" name is in pretty common usage.

[headius (Charles Nutter)]

• Assignee set to akr (Akira Tanaka)
• Target version set to Ruby 2.1.0

This is now provided by #8658 in 2.1, but according to matz in the developer meeting notes linked there, Process::clock_gettime is not considered "spec".

So I'm not sure if this can be closed or not...we need to decide.

Optimistically assigning to akr.

[akr (Akira Tanaka)]

• Assignee deleted (akr (Akira Tanaka))

I can not decide Ruby's specification.

[naruse (Yui NARUSE)]

• Assignee set to matz (Yukihiro Matsumoto)

headius (Charles Nutter) wrote:

This is now provided by #8658 in 2.1, but according to matz in the developer meeting notes linked there, Process::clock_gettime is not considered "spec".

So I'm not sure if this can be closed or not...we need to decide.

Optimistically assigning to akr.

Process.clock_gettime is low level API; it is different from this high level API.
On the meeting, matz considered this high level API didn't get consensus yet.

[headius (Charles Nutter)]

It would not be hard, given the guarantee that CLOCK_REALTIME works across platforms, to provide a #timestamp that is always CLOCK_REALTIME.

Can we do that, so we have an official API?

[naruse (Yui NARUSE)]

headius (Charles Nutter) wrote:

It would not be hard, given the guarantee that CLOCK_REALTIME works across platforms, to provide a #timestamp that is always CLOCK_REALTIME.

Can we do that, so we have an official API?

We can but whether is is official or not is matz's decision.
Moreover Process.clock_gettime is different issue from this ticket.

[kosaki (Motohiro KOSAKI)]

• Related to Feature #12173: `Time#till_now` added

[hsbt (Hiroshi SHIBATA)]

• Project changed from 14 to Ruby master
• Target version deleted (Ruby 2.1.0)