Misc #14632: [ANN] git.ruby-lang.org

• Status: Closed
• Assignee: hsbt (Hiroshi SHIBATA)

[hsbt (Hiroshi SHIBATA)]

I prepared git.ruby-lang.org. It's official ruby repository of Git.

https://git.ruby-lang.org/ruby.git/

This repository is read-only yet. I start to migrate svn to git step by step.

1. Migrate toolchain like tool/make-snapshot to git.ruby-lang.org from svn.ruby-lang.org.
2. Improve cgit and access configuration.
3. To support git repository for bug.ruby-lang.org. PS. redmine could not support remote git repository.
4. Make write-access mode to git.ruby-lang.org. and make the read-only mode to svn.ruby-lang.org.

[hsbt (Hiroshi SHIBATA)]

• Related to Misc #10547: How to move the ruby project to git added

[vo.x (Vit Ondruch)]

Wonderful. Thx for working on this.

Have you considered to use Pagure 1 as git fronted instead of cgit?

[normalperson (Eric Wong)]

[e-mail] wrote:

I prepared git.ruby-lang.org. It's official ruby repository of Git.

Thank you!

3. To support git repository for bug.ruby-lang.org. PS. redmine could not support remote git repository.

I guess you will need to keep a mirror on the redmine server?
Sorry, I'm not familiar with redmine setup; but I know git :)

[normalperson (Eric Wong)]

[e-mail] wrote:

Have you considered to use Pagure 1 as git fronted instead of cgit?

Pagure would be more a replacement for redmine than a replacement
for cgit, I think. In any case, I prefer loose coupling between
version control and messaging platform.

Dne 26.3.2018 v 23:30 Eric Wong napsal(a):

[e-mail] wrote:

Have you considered to use Pagure 1 as git fronted instead of cgit?

Pagure would be more a replacement for redmine than a replacement
for cgit, I think.

It might be of course. It depends which features you enable. But for
example for Fedora sources 1, Pagure is used as git frontend allowing
browsing and accepting PR, managing users, etc, but Bugzilla is used as
a bug tracker. I can imagine similar setup for Ruby, where Redmine is
still used for bug reporting.

Vít

1 https://src.fedoraproject.org/rpms/ruby
[2] https://bugzilla.redhat.com/

[shevegen (Robert A. Heiler)]

\o/

Hiroshi for president as well!

\o~

~o/

\o/

hello,

On 2018-03-26 11:47:38 +0000, [e-mail] wrote:

Issue #14632 has been reported by hsbt (Hiroshi SHIBATA).

---

Misc #14632: [ANN] git.ruby-lang.org
https://bugs.ruby-lang.org/issues/14632
[…]
I prepared git.ruby-lang.org. It's official ruby repository of Git.

https://git.ruby-lang.org/ruby.git/

Thanks a lot for this, I'm learning SVN, but still more comfortable
with git.

I had a few issues to retrieve the new repository:

$ git clone git://git.ruby-lang.org/ruby
Cloning into 'ruby'...
load: 0.18 cmd: git 61356 [connec] 4.90r 0.00u 0.01s 0% 3872k
fatal: unable to connect to git.ruby-lang.org:
git.ruby-lang.org[0: 52.192.80.219]: errno=Operation timed out
zsh: exit 128 git clone git://git.ruby-lang.org/ruby

$ nc -v git.ruby-lang.org 9418
nc: connect to git.ruby-lang.org port 9418 (tcp) failed: Operation timed out
zsh: exit 1 nc -v git.ruby-lang.org 9418

I could not connect using git protocol, but maybe it's normal and it
won't be supported? I then attempted HTTPS, but it was a bit slow:

$ time git clone https://git.ruby-lang.org/ruby.git ruby
Cloning into 'ruby'...
load: 0.36 cmd: git-remote-https 61502 [select] 20.97r 0.45u 0.67s 3% 12512k
load: 0.22 cmd: git-remote-https 61502 [select] 51.96r 0.97u 1.41s 2% 15456k
load: 0.96 cmd: git 61525 [running] 60.56r 84.08u 6.33s 100% 178776k
load: 0.96 cmd: git-remote-https 61502 [running] 193.68r 2.17u 2.95s 1% 15996k
load: 0.25 cmd: git-remote-https 61502 [select] 308.69r 6.26u 5.17s 5% 121764k
Checking connectivity: 396310, done.
git clone https://git.ruby-lang.org/ruby.git ruby 108.45s user 14.47s system 26% cpu 7:46.62 total

I'm not complaining, I can of course wait and won't clone it
everyday. But this surprised me a little, am I doing something wrong?
Is it the correct way to fetch this new repository?

Thanks again.

--
Thibault Jouan

[hsbt (Hiroshi SHIBATA)]

I could not connect using git protocol, but maybe it's normal and it
won't be supported? I then attempted HTTPS, but it was a bit slow:

git.r-l.o only supports https protocol now. And I try to improve it clone speed.

[hsbt (Hiroshi SHIBATA)]

Hi all.

I added git.ruby-lang.org to this redmine project.

You can refer git ref like https://bugs.ruby-lang.org/projects/ruby-trunk/repository/ruby-git/revisions/5896936f2ca0c16a68edb557a122125653fa89a7
Redmine expand commit:ruby-git|5896936f2ca0c16a68edb557a122125653fa89a7 to above URL now. (This long syntax is not necessary when git repository was promoted default repository. )

I may change repository identifier (ruby-git ) until completing git migration.

[normalperson (Eric Wong)]

[e-mail] wrote:

git.r-l.o only supports https protocol now. And I try to
improve it clone speed.

If you haven't already, enable repack.writeBitmaps (and repack)
to speed up the counting objects phase.

[hsbt (Hiroshi SHIBATA)]

If you haven't already, enable repack.writeBitmaps (and repack)

Thanks Eric. I invoked repack and set repack.writeBitmaps to enable on git.r-l.o.

[hsbt (Hiroshi SHIBATA)]

Current status is here:

1. Migrate toolchain like tool/make-snapshot to git.ruby-lang.org from svn.ruby-lang.org.

Not yet, We need to update make-snapshot and backport tool for stable branches.

2. Improve cgit and access configuration.

Done.

3. To support git repository for bug.ruby-lang.org. PS. redmine could not support remote git repository.

Done.

4. Make write-access mode to git.ruby-lang.org. and make the read-only mode to svn.ruby-lang.org.

Not yet.

And We need to configure git-hook for auto-style by svn bot and github sync tool.

[k0kubun (Takashi Kokubun)]

To remind us the current status, let me list up the details of remaining tasks for git migration:

Preparations

• chkbuild: change all RubyCI's checkout source from svn to git https://github.com/ruby/chkbuild/pull/42
  • And fix the problem showing git revision on RubyCI

Operations

• Make git.ruby-lang.org writable (just announce it), and make svn.ruby-lang.org read-only.
  • Also prohibit force-push to git.ruby-lang.org
• Uncomment auto-style and update-version in hooks/post-receive.sh

After repository migration to git

• Use a git commit hash instead of SVN revision in RUBY_REVISION, RUBY_DESCRIPTION
• Add more git support in:
  • tool/vcs.rb:
    • tool/change_maker.rb
    • tool/file2lastrev.rb
    • tool/make-snapshot (should be ready before 2.7.0-preview1?)
  • Git support to backport to future ruby_2_7 branch:
    • tool/redmine-backporter.rb
    • tool/merger.rb
• Fix broken git diff on redmine

Caveats

• Do not create a topic branch on cgit.
• GitHub repository will still be just a mirror. Don't push "Merge pull request" button.
• ruby_2_4, ruby_2_5, ruby_2_6 branches will continue to use SVN. Don't push anything to these branches on cgit.

[k0kubun (Takashi Kokubun)]

Progress:

• I migrated svn hooks repository to https://github.com/ruby/ruby-commit-hook, and supported git in all features.
• Backport tools (tool/merger.rb and tool/redmine-backporter.rb) supported to specify git revision in r67693 and r67697.
• Updated svn-related information to git version in:
  • https://bugs.ruby-lang.org/projects/ruby/wiki/DeveloperHowto
  • https://bugs.ruby-lang.org/projects/ruby/wiki/DeveloperHowtoJa
  • https://bugs.ruby-lang.org/projects/ruby/wiki/CommitterHowto
  • https://bugs.ruby-lang.org/projects/ruby/wiki/CommitterHowtoJa
  • https://bugs.ruby-lang.org/projects/ruby/wiki/HowToBackport

I believe there's no more immediate blockers other than "Preparations" and "Operations" written in the above comment. I'll wait @hsbt (Hiroshi SHIBATA) for those tasks.

[hsbt (Hiroshi SHIBATA)]

@k0kubun (Takashi Kokubun) Thanks to your works.

I enabled and disabled some changes on our repository.

Operations

• Make git.ruby-lang.org writable (just announce it), and make svn.ruby-lang.org read-only.

done. I disabled the writable operation of trunk branch on svn.ruby-lang.org

Also prohibit force-push to git.ruby-lang.org

done. I added denyNonFastForwards = true on git.ruby-lang.org.

Uncomment auto-style and update-version in hooks/post-receive.sh

done. https://github.com/ruby/ruby-commit-hook/commit/0e8fd2abc43e98a703c8bc8d3ec4fe90305409ac

To. the Ruby committers.

Please update your origin variable of git working directory to [e-mail]:ruby.git. If you don't have the writable access, Please share your svn account and the public key of ssh.

I will announce Moving to git from svn later.

[shyouhei (Shyouhei Urabe)]

hsbt (Hiroshi SHIBATA) wrote:

done. I disabled the writable operation of trunk branch on svn.ruby-lang.org

Next time tell us BEFORE you disable something, OK?

[hsbt (Hiroshi SHIBATA)]

Next time tell us BEFORE you disable something, OK?

I will make an effort next time. Can you join ruby-lang slack team again?
We discuss the internal workflow on this slack.

[shyouhei (Shyouhei Urabe)]

hsbt (Hiroshi SHIBATA) wrote:

Next time tell us BEFORE you disable something, OK?

I will make an effort next time. Can you join ruby-lang slack team again?
We discuss the internal workflow on this slack.

I now feel I am excluded from the community because I was not on the chat. Does this mean anyone must join the slack if they have to be considered as a committer?

[naruse (Yui NARUSE)]

In this ticket, there's a list of pre and post tasks.
You can always add "pre-announcement" and "post-announcement" in these 3 months.

About git migration, investigation and organizing tasks themselves are also hard work.
I'm very appreciate for people who worked hard for them.

Anyway chat is out of this topic.

[shyouhei (Shyouhei Urabe)]

naruse (Yui NARUSE) wrote:

Anyway chat is out of this topic.

Yes please. Don't let the chat involve this kind of things.

[k0kubun (Takashi Kokubun)]

Remaining tasks related to Git migration

• Add more git support in:
  • Git support to backport to future ruby_2_7 branch:
    • tool/merger.rb
• Discuss the timing to rename the branch "trunk" to "master"? (#15782)

[mame (Yusuke Endoh)]

k0kubun (Takashi Kokubun) wrote:

• Verify committer email from SSH user (@mame (Yusuke Endoh))

Done.

• I've created a script named check-email.rb which includes the correspondence table between git ssh key (SVN account name) and committer's email.
• This script is run on the git server when pushed, and checks if the used ssh key is actually associated with committer's email in each pushed commit.
• Even if the used key is not listed in the script, you can do push commits until the last of 2019, and the association of ssh key and email is logged. I will occasionally add the logged association to the script manually.
• In 2020 or later, you will NOT be able to push commits if your email is not listed in the script. Please send PR to github:ruby-commit-hook or contact on @hsbt (Hiroshi SHIBATA), @naruse (Yui NARUSE), @k0kubun (Takashi Kokubun), or me.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Assigned to Closed

I announced https://www.ruby-lang.org/en/news/2019/04/23/move-to-git-from-svn/

We still have some issues. But I will close this.

If you find the issues, Please file them to bugs.ruby-lang.org or https://github.com/ruby/b.r-l.o especially redmine.

[k0kubun (Takashi Kokubun)]

Today and yesterday I updated ruby-commit-hook to deal with race condition and corner cases of notifications, like pushing multiple commits at once. Now ruby-commit-hook should be correctly working for much more situations.

@nobu (Nobuyoshi Nakada) and I have fixed tool/*.rb to support Git. I think either of us made Git support for all tool/*.rb which needed changes. Today I changed tool/redmine-backporter.rb and tool/merger.rb to support backporting changes to Git repository for the future 2.7+ stable branches.

I believe we completed Git migration of Ruby repository. Please file another ticket if you find something or a bug in backport tools. Thank you.

[k0kubun (Takashi Kokubun)]

• Related to Misc #15843: Make "trunk" a symbolic-ref of "master" on git.ruby-lang.org added

[k0kubun (Takashi Kokubun)]

• Related to Misc #16094: Allow only "Rebase and merge" or "Squash and merge" on GitHub master branch, and sync it on git.ruby-lang.org update hook added

[mame (Yusuke Endoh)]

All the Ruby committers,

This is an important reminder: You will not be able to push any change to git.ruby-lang.org after 1st Jan. 2020 unless your committer email address is not listed in https://github.com/ruby/ruby-commit-hook/blob/master/config/email.yml.

This does never mean your commit bit will be revoked; even after the date, we can add your email address to the file and then you will be able to push a change. But I'd like to inform you in advance, and recommend you to add your email address before the date.

If you want to add your email address to the file, send a pull request to the GitHub repository, or contact on @hsbt (Hiroshi SHIBATA), @naruse (Yui NARUSE), @k0kubun (Takashi Kokubun), or me in any way you like (e.g., reply to this ticket, send a mail to us, etc.)

Japanese:

コミッタのみなさん、

重要なご連絡です。次のURLのリストにあなたのメールアドレスが載っていないと、2020年1月1日以降、あなたはgit.ruby-lang.orgに変更をpushできなくなります（コミットできなくなります）。

https://github.com/ruby/ruby-commit-hook/blob/master/config/email.yml

コミット権限が失効するというわけではありません。期日のあとでも、そのリストにメールアドレスを追加することはできますし、そうすればコミットできるようになります。事前にお伝えしておくためのご連絡です。できれば期日の前までにメールアドレスを追加することを推奨いたします。

メールアドレスをこのファイルに追加したい場合は、GitHubリポジトリにpull requestを送るでもよいし、 @hsbt (Hiroshi SHIBATA), @naruse (Yui NARUSE), @k0kubun (Takashi Kokubun), または私に任意の手段でご連絡ください（このチケットに返答するでも、メールを送るでも、お好きな方法で）。

[duerst (Martin Dürst)]

A very small but important correction:

mame (Yusuke Endoh) wrote:

All the Ruby committers,

This is an important reminder: You will not be able to push any change to git.ruby-lang.org after 1st Jan. 2020 unless your committer email address is not listed in https://github.com/ruby/ruby-commit-hook/blob/master/config/email.yml.

This is an important reminder: You will not be able to push any change to git.ruby-lang.org after 1st Jan.
2020 unless your committer email address >IS< listed in https://github.com/ruby/ruby-commit-hook/blob/master/config/email.yml.

This does never mean your commit bit will be revoked; even after the date, we can add your email address to the file and then you will be able to push a change. But I'd like to inform you in advance, and recommend you to add your email address before the date.

If you want to add your email address to the file, send a pull request to the GitHub repository, or contact on @hsbt (Hiroshi SHIBATA), @naruse (Yui NARUSE), @k0kubun (Takashi Kokubun), or me in any way you like (e.g., reply to this ticket, send a mail to us, etc.)

[mame (Yusuke Endoh)]

@duerst (Martin Dürst) Yes, thanks!

[normalperson (Eric Wong)]

@hsbt (Hiroshi SHIBATA): please add [e-mail]
(I don't have a GH account, will not use JS or agree to their Terms of Service)

Btw, ruby-core -> redmine mail is still broken:
[ruby-core:96254] [Feature #16131] (posted recently)
original report: [ruby-core:95662]

[mame (Yusuke Endoh)]

Hi @normal , I've added your email address to the list.