Feature #14602
openVersion of dig that raises error if a key is not present
Description
Currently, if I have a hash like this:
{
:name => {
:first => "Ariel",
:last => "Caplan"
}
}
and I want to navigate confidently and raise a KeyError if something is missing, I can do:
hash.fetch(:name).fetch(:first)
Unfortunately, the length of the name, combined with the need to repeat the method name every time, means most programmers are more likely to do this:
hash[:name][:first]
which leads to many unexpected errors.
The Hash#dig method made it easy to access methods safely from a nested hash; I'd like to have something similar for access without error protection, and I'd think the most natural name would be Hash#dig!. It would work like this:
hash = {
:name => {
:first => "Ariel",
:last => "Caplan"
}
}
hash.dig!(:name, :first) # => Ariel
hash.dig!(:name, :middle) # raises KeyError (key not found: :middle)
hash.dig!(:name, :first, :foo) # raises TypeError (String does not have #dig! method)
Updated by shevegen (Robert A. Heiler) over 6 years ago
I think this may be somewhat problematic since it does not appear
to fit to other methods that end with a "!", such as .chop() versus
.chop!() for a String or .map() versus .map!() for an Array.
In the past I thought that "!" would mean mostly "modify in place",
but matz wrote somewhere on the bug tracker that it is more meant
as a "caution" indicator to the ruby user.
Another problem is, I think, that your suggestion of .dig!() does
something that .dig() itself isn't doing (raising multiple
different errors). But that may just be me, perhaps others see
no problem - at the end of the day you'd only have to convince
matz anyway. :)
Updated by duerst (Martin Dürst) over 6 years ago
Would a keyword parameter to dig work for you?
E.g. hash.dig!(:name, :middle, raise_error: true)
or something similar.
Updated by amcaplan (Ariel Caplan) over 6 years ago
shevegen (Robert A. Heiler) wrote:
I think this may be somewhat problematic since it does not appear
to fit to other methods that end with a "!", such as .chop() versus
.chop!() for a String or .map() versus .map!() for an Array.In the past I thought that "!" would mean mostly "modify in place",
but matz wrote somewhere on the bug tracker that it is more meant
as a "caution" indicator to the ruby user.Another problem is, I think, that your suggestion of .dig!() does
something that .dig() itself isn't doing (raising multiple
different errors). But that may just be me, perhaps others see
no problem - at the end of the day you'd only have to convince
matz anyway. :)
You have a good point about the bang methods often signifying an in-place operation rather than an error-prone one; the latter is more of a Rails convention than a Ruby one, and I mixed things up. Thank you for pointing that out. Mea culpa.
Ideally, we'd just have Hash#fetch
take an arbitrary number of arguments, each representing another layer of depth. However, that option is closed off due to backwards compatibility issues, since it already takes an optional second argument representing a default
return value in case the item isn't found.
So it sounds like we need a new method name. Perhaps something like #deep_fetch
would work, but I'll have to think on it more to see if I can come up with something better. EDIT: Apparently there's already a deep_fetch
gem that does exactly this, which might indicate that the name #deep_fetch
would be understood intuitively by the community.
In terms of the multiple errors: This isn't actually new, and in fact I copied the current behavior of Hash#dig
with the exception of adding the KeyError
:
hash = {
:name => {
:first => "Ariel",
:last => "Caplan"
}
}
hash.dig(:name, :first) # => Ariel
hash.dig(:name, :middle) # => nil
hash.dig(:name, :first, :foo) # raises TypeError (String does not have #dig method)
Updated by amcaplan (Ariel Caplan) over 6 years ago
duerst (Martin Dürst) wrote:
Would a keyword parameter to dig work for you?
E.g.
hash.dig!(:name, :middle, raise_error: true)
or something similar.
I appreciate the thought. I personally would be more likely to do hash.fetch(:name).fetch(:middle)
instead of adding a keyword argument to #dig
, unless the list was extremely long (probably at least 4 consecutive keys), which I'd suspect is unusual enough that it's not worth adding to Ruby core for that unusual case.
Updated by k0kubun (Takashi Kokubun) over 5 years ago
- Is duplicate of Feature #12282: Hash#dig! for repeated applications of Hash#fetch added
Updated by k0kubun (Takashi Kokubun) over 5 years ago
- Has duplicate Feature #15563: #dig that throws an exception if a key doesn't exist added
Updated by robb (Robb Shecter) over 4 years ago
amcaplan (Ariel Caplan) wrote:
The Hash#dig method made it easy to access methods safely from a nested hash; I'd like to have something similar for access without error protection, and I'd think the most natural name would be Hash#dig!.
FYI, I've implemented this as a gem: https://github.com/dogweather/digbang
Updated by jaredbeck (Jared Beck) over 2 years ago
I personally don't mind dig!
. I interpret the !
as a general sign of caution, rather than some meaning specific to data structures (ie. self-modification). But, if we can't have dig!
, how about fetch_dig
or dig_fetch
?
Whatever we call it, it's a good idea and should be in standard ruby. IMO it's too small to be a gem. We don't want to live in a "left-pad" world. :)
Updated by fpsvogel (Felipe Vogel) over 2 years ago
For me this is a nice shortcut to safely access values in a large config hash. So I would use it if it became part of Ruby core.
I like the name dig!
because it's short, but if that has too much of a Rails flavor rather than Ruby, then deep_fetch
and dig_fetch
seem like fine names too.
I recently wrote about some alternatives for this with benchmarks and other considerations, including the two gems mentioned above as well as custom implementations, at https://fpsvogel.com/posts/2022/ruby-hash-dot-syntax-dig-performance-benchmarks#dig-with-errors
Updated by zverok (Victor Shepelev) over 2 years ago
Just a bit of "design space" analysis:
- I think
dig!
is unusual for core Ruby. A lot of Rubyists are used that in Rails pairs likefind_by
/find_by!
are raising/non-raising, but I don't remember any Ruby core API using this convention - I don't believe the keyword argument is expressive enough. The "visual structure" of the
dig
signature includes multiple values of user data (and the list of values might be of arbitrary length), so the option at the end of arguments is a) not visible enough and b) not immediately intuitively obvious if it isn't part of user's data - In Hash, we already have at least two examples of using
fetch
in a sense "get the value or fail":#[]
vs#fetch
and#values_at
vs#fetch_values
. It seems like it gives enough precedent to look at the "fetch"-based naming, and it seems likefetch_dig
, while grammatically not ideal, would be guessable enough, based on existing experience.
Updated by jeremyevans0 (Jeremy Evans) over 2 years ago
duerst (Martin Dürst) wrote in #note-2:
Would a keyword parameter to dig work for you?
E.g.
hash.dig!(:name, :middle, raise_error: true)
or something similar.
Keyword approach is not backwards compatible, because keywords are currently treated as positional arguments:
{:name=>{:middle=>{{:raise_error=>true}=>2}}}.dig(:name, :middle, raise_error: true)
# => 2
Updated by nobu (Nobuyoshi Nakada) over 2 years ago
[:name, :middle].inject(hash, :fetch) # raises KeyError (key not found: :middle)
Updated by shyouhei (Shyouhei Urabe) over 2 years ago
nobu (Nobuyoshi Nakada) wrote in #note-12:
[:name, :middle].inject(hash, :fetch) # raises KeyError (key not found: :middle)
Doesn't interface with Arrays.
Updated by mame (Yusuke Endoh) over 2 years ago
shyouhei (Shyouhei Urabe) wrote in #note-13:
Doesn't interface with Arrays.
[0, 0, 0].inject([[[:foo]]], :fetch) #=> :foo
?
I don't think @nobu (Nobuyoshi Nakada) is serious about the idiom, though.
Updated by shyouhei (Shyouhei Urabe) over 2 years ago
mame (Yusuke Endoh) wrote in #note-14:
shyouhei (Shyouhei Urabe) wrote in #note-13:
Doesn't interface with Arrays.
[0, 0, 0].inject([[[:foo]]], :fetch) #=> :foo
?I don't think @nobu (Nobuyoshi Nakada) is serious about the idiom, though.
Hmm, sorry. I was confusing.
Updated by Eregon (Benoit Daloze) over 2 years ago
There is also this blog post discussing about dig_fetch
, which seems very similar: https://shopify.engineering/dig-fetch-truffleruby
message = data.dig_fetch(:response, :message) { IdentityObject.new }
# instead of
message = data.fetch(:response, {}).fetch(:message, IdentityObject.new)
So it still allows a default value if any part is missing when provided a block, and raises an exception otherwise.
Updated by matz (Yukihiro Matsumoto) over 2 years ago
It seems to be a nice idea. But is deep_fetch
is the best name for it?
Matz.
Updated by byroot (Jean Boussier) over 2 years ago
A couple other ideas I had:
lookup
pick
traverse
Updated by st0012 (Stan Lo) over 2 years ago
How about dig_for
? If we’re digging for something, it kinda makes sense to raise an exception if it’s not there.
Updated by mollemoll (Jonas Molander) over 2 years ago
Some ideas:
reveal
uncover
unfold
dive
enlight
scoop
reel
Updated by p8 (Petrik de Heus) over 2 years ago
Maybe:
fetch_each(:name, :first)
fetch_all(:name, :first)
fetch_tail(:name, :first)
fetch_end(:name, :first)
# or with dig_
dig_each(:name, :first)
dig_tail(:name, :first)
dig_all(:name, :first)
dig_end(:name, :first)
dig_bottom(:name, :first)
dig_deep(:name, :first)
dig_down(:name, :first)
# or maybe
delve(:name, :first)
drill(:name, :first) # as in drill down
Updated by olivierlacan (Olivier Lacan) over 2 years ago
Considering dig
as an operation that can be done manually (with fingers) and address exceptional situations (unexpected objects) more subtly, I would suggest shovel
which is a more blunt instrument which tends to raise a notable exception sound (clang!) when it hits an unexpected object.
hash = {
name: {
first: "Ariel",
last: "Caplan"
}
}
hash.dig(:name, :first) # => Ariel
hash.dig(:name, :middle) # => nil
hash.shovel(:name, : first) # => Ariel
hash.shovel(:name, :middle) # => KeyError (key not found: :middle)
There's a slight semantic overlap with the Hash shovel operator (<<
) which could be an issue.
Updated by amcaplan (Ariel Caplan) over 2 years ago
We can think of this as either a variation of fetch
or a variation of dig
. Ultimately it's both, of course, just depends how you look at it.
If we think of it as fetch
-based, deep_fetch
would be OK but we also might go with something that really describes quite literally what it does, which is fetch
recursively. So, fetch_recursive
or rfetch
(think of Array#bsearch
as prior art - though of course it's not 100% comparable) might be the way to go.
If we take the dig
-based perspective, it's dig
but non-permissive. So dig_strict
might be the most literal way of explaining what it does.
Rather than advocating strongly for 1 specific word, I'd just gently recommend that we avoid introducing more dig-like verbs. While seasoned Rubyists will know the difference, newcomers won't have any obvious reason to assume that dig
differs from traverse
in strictness. At least there's a convention for fetch
vs []
which already exists and we can use it to avoid introducing more new language/concepts.
Updated by duerst (Martin Dürst) over 2 years ago
amcaplan (Ariel Caplan) wrote in #note-23:
We can think of this as either a variation of
fetch
or a variation ofdig
. Ultimately it's both, of course, just depends how you look at it.
Or maybe we can think it as a combination of dig
with fetch
. Then what about dig_fetch
or fetch_dig
? These names don't look very natural, but it's easy to understand what they are about.
Updated by ufuk (Ufuk Kayserilioglu) over 2 years ago
My humble suggestion would be Hash#retrieve
. The operation tries to retrieve a value in the depths of a hash and if it comes back empty handed, that is an error.
hash = {
name: {
first: "Ariel",
last: "Caplan"
}
}
hash.dig(:name, :first) # => Ariel
hash.dig(:name, :middle) # => nil
hash.retrieve(:name, :first) # => Ariel
hash.retrieve(:name, :middle) # => KeyError (key not found: :middle)
The word "retrieve" is a synonym for "fetch" in English, and has deep roots in CS terminology related to "data retrieval" and similar.
Updated by zverok (Victor Shepelev) over 2 years ago
I fully agree with @duerst (Martin Dürst) in #14602#note-24:
maybe we can think it as a combination of
dig
withfetch
. Then what aboutdig_fetch
orfetch_dig
? These names don't look very natural, but it's easy to understand what they are about.
First, we already have examples of fetch
-based naming: not only #fetch
itself as a variation of #[]
, but also #fetch_values
as a variation of #values_at
, so there is a precedent for recognizability
Second, I value short one-word names, so all the witty options like #shovel
and #retrieve
are nice, but I am afraid that when we have a variation of a known method in an API established long ago, introducing completely new word into Ruby would be a false move. Imagine you started to read code and met with #retrieve
(or #shovel
) for the first time. There is nothing that might help you to understand what it does; one verb that "a bit resembles dig
" is not suggestive enough.
Third, deep_fetch
is somewhat suggestive, but the problem "it behaves like dig
, but the name logic is nothing like dig
" stands. Maybe if it would a pair of, IDK, #deep_fetch
and #deep_get
it might've been tolerable, but now is too late for that, everybody has used to #dig
.
fetch_dig
, OTOH, is reasonably short, clearly suggests the meaning, and follows the logic of other methods existing.
Updated by sawa (Tsuyoshi Sawada) over 2 years ago
What about simply allowing fetch
to take multiple arguments?
If there are more than one argument, always interpret the last one as the default value unless when there is a block, in which case, the block is evaluated if a key is missing somewhere in the path.
You would need to explicitly write a block if you want to raise an error, but I believe your intention is not to let the raised error go through all the way to the top level to end the program; you intend to catch that error somewhere, and do something with it, right? Then, you can instead write that routine in the block from the beginning.
(1) No change to present behavior:
hash.fetch(:name, :first) # => {:first => "Ariel", :last => "Caplan"}
(2) Perhaps error prone use cases, but these are suited for dig
, so do not use fetch
in practice in such cases, and it would not be a problem:
hash.fetch(:name, :first, nil) # => "Ariel"
hash.fetch(:name, :middle, nil) # => `nil`
(3) Explicitly raise an error in a block:
hash.fetch(:name, :first){|key| raise KeyError} # => "Ariel"
hash.fetch(:name, :middle){|key| raise KeyError} # !> KeyError
(4) Or, write a routine:
hash.fetch(:name, :middle){|key| process_missing_key(key)}
Updated by sinsoku (Takumi Shotoku) about 1 year ago
I agree with #14602#note-23.
I think a fetch-based or dig-based name would be better, since the function can be inferred from the method name.
Personally, I feel that deep_fetch is a simple and nice name.
I often use deep_dup and deep_merge in Rails apps and are familiar with the method name deep_*
.
I found fetch_dig
and dig_fetch
to be a little strange in that they have verbs lined up.
I think dig_strict
is a good name, but I prefer deep_fetch because deep
has fewer characters and is simpler than strict
.
Updated by Eregon (Benoit Daloze) about 1 year ago
I like deep_fetch
too.
Updated by mame (Yusuke Endoh) about 1 year ago
We discussed at the dev meeting but did not reach a conclusion. But there has been some progress.
- @akr (Akira Tanaka) said we shouldn't add new vocabulary, and Matz agreed.
- There is
Array#fetch
which throws an exception when it is not found, so we want to use this vocabulary:fetch_*
or*_fetch
- Matz didn't like any of the proposed
fetch_each
fetch_all
fetch_tail
fetch_end
- Matz didn't like any of the proposed
- Matz prefers
dig
orrecursive
todeep
to represent this behavior.-
deep
sounds to be an operation on the entire data structure, due to the influence of deep_copy.
-
- Matz was interested in how
dig_fetch
orfetch_dig
would sound to English native speakers. - Matz also mentioned
recursive_fetch
, but said it seemed a bit long.
Updated by Eregon (Benoit Daloze) about 1 year ago
mame (Yusuke Endoh) wrote in #note-30:
deep
sounds to be an operation on the entire data structure, due to the influence of deep_copy.
It actually is a deep operation, on the entire data structure/multiple objects, exactly like deep_copy would.
So it seems a perfect match, no?
Updated by mame (Yusuke Endoh) about 1 year ago
As far as I understand, Matz expects that, for a tree structure:
-
dig_*
touches only nodes on a specific path, -
deep_*
touches all nodes.
Updated by edmz (Ed Mz) 10 months ago
Following what @mame (Yusuke Endoh) explained about Matz expectations, I would like to (humbly) suggest another option:
dig_expected
(and a my +1 to dig_strict
)
Updated by jordan-brough (Jordan Brough) 3 months ago
Matz was interested in how dig_fetch or fetch_dig would sound to English native speakers.
As a native english speaker, "dig_fetch" sounds natural and makes sense to me.