Ruby » Ruby master

Feature #19058

Updated by ioquatix (Samuel Williams) over 1 year ago

There are many instances of programs using globals, thread locals and fiber locals for shared state. Unfortunately these programs often have bugs or unusual behaviour when objects are used on different threads or fibers. 

 Here is a simple example of the kind of problem that can occur: 

 ```ruby 
 class Users 
   
	 def each 
     
		 return to_enum unless block_given? 
     
		 p each: Fiber.current 
     
		 yield "A" 
     
		 yield "B" 
     
		 yield "C" 
   
	 end 
 end 

 p Users.new.each.zip(Users.new.each) 
 ``` 

 When the enumeration depends on a finer-local connection instance (e.g. `Thread.current[:connection]`) it could unexpectedly break the operation because within the enumerate, the fiber is different leading to missing connection. 

 In all these cases, the problem can be solved by not relying on implicit/invisible state. Unfortunately, many programs take advantage of process (global), thread or fiber local variables to create more ergonomic interfaces, e.g. 

 ```ruby 
 DB.connect(username, password, host) 

 DB.query("SELECT * FROM BUGS"); # implicit dependency on connection. 
 ``` 

 Ruby provides several, somewhat confusing options. 

 ```ruby 
 Thread.current[:x] # Fiber local. 
 Thread.current.thread_variable_get(:x) # Thread local. 

 class Thread 
   attr :x 
 end 

 Thread.current.x # thread local 

 class Fiber 
   attr :x 
 end 

 Fiber.current.x # fiber local 
 ``` 

 Over the years there have been multiple issues about the above behaviour: 

 - https://bugs.ruby-lang.org/issues/1717 
 - https://bugs.ruby-lang.org/issues/7097 
 - https://bugs.ruby-lang.org/issues/8215 
 - https://bugs.ruby-lang.org/issues/13893 
 - Rails is also working around this issue: https://github.com/rails/rails/pull/43596 

 The heart of the issue is: there should be a consistent and convenient way to define state attached to the current execution context, which gets inherited by child execution contexts. Essentially: 

 ```ruby 
 let(x: 10) do 
   # In every execution context created within this block, unless otherwise changed, `x` is bound to the value 10. That means, threads, fibers, etc. 
 end 
 ``` 

 This is sometimes referred to as dynamic scope. This proposal is to introduce a similar dynamic scope for fiber inheritable attributes. 

 ## Proposal 

 We have several units of execution in Ruby, implicitly a process, which has threads which has fibers. Internally, Ruby has an "execution context". Each fiber has an execution context. Each thread has a main fiber and execution context. I propose to introduce an interface for defining a per-fiber context which is semantically very similar to dynamic variables. True dynamic variables are stack frame scoped, but my proposal doesn't go that far. This proposal is similar to Kotlin's Coroutine Contexts <https://kotlinlang.org/docs/coroutine-context-and-dispatchers.html>. 

 ```ruby 
 require 'fiber' 

 # Compatible shim to introduce proposed behaviour: 
 class Fiber 
   
	 module Inheritable 
     
		 def initialize(...) 
       
			 super(...) 
      
       
			
			 self.inherit_attributes_from(Fiber.current) 
     
		 end 
    
     
		
		 def self.prepended(klass) 
       
			 klass.extend(Singleton) 
       
			 klass.instance_variable_set(:@inheritable_attributes, Hash.new) 
     
		 end 
    
     
		
		 module Singleton 
       
			 def inheritable(key, default: nil) 
         
				 @inheritable_attributes[:"@#{key}"] = default 
       
			 end 
      
       
			
			 def inheritable_attributes 
         
				 @inheritable_attributes 
       
			 end 
     
		 end 
    
     
		
		 def inherit_attributes_from(fiber) 
       
			 self.class.inheritable_attributes.each do |name, default| 
         
				 value = fiber.instance_variable_get(name) || default 
         
				 self.instance_variable_set(name, value) 
       
			 end 
     
		 end 
   
	 end 
 end 

 unless Fiber.respond_to?(:inheritable) 
   
	 Fiber.prepend(Fiber::Inheritable) 
 end 
 ``` 

 This allows you to write the following code: 

 ```ruby 
 class Fiber 
   inheritable attr_accessor :connection 
 end 

 # When lazy enumerator creates internal fiber, the connection and related state will be inherited correctly: 
 p User.first(100).find_each.zip(Post.first(100).find_each) 
 ``` 

 Some open questions: 

 - Should we introduce the same interface for Thread? 
 - (or) Should Thread.new's main fiber inherit from the current Fiber? 

Back