Project

General

Profile

Misc #14632

[ANN] git.ruby-lang.org

Added by hsbt (Hiroshi SHIBATA) about 1 year ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
[ruby-core:86304]

Description

I prepared git.ruby-lang.org. It's official ruby repository of Git.

https://git.ruby-lang.org/ruby.git/

This repository is read-only yet. I start to migrate svn to git step by step.

  1. Migrate toolchain like tool/make-snapshot to git.ruby-lang.org from svn.ruby-lang.org.
  2. Improve cgit and access configuration.
  3. To support git repository for bug.ruby-lang.org. PS. redmine could not support remote git repository.
  4. Make write-access mode to git.ruby-lang.org. and make the read-only mode to svn.ruby-lang.org.

Related issues

Related to Ruby trunk - Misc #10547: How to move the ruby project to gitRejectedActions
Related to Ruby trunk - Misc #15843: Make "trunk" a symbolic-ref of "master" on git.ruby-lang.orgAssignedActions

Associated revisions

Revision 67693
Added by k0kubun (Takashi Kokubun) 2 months ago

tool/merger.rb: Support fetching patch from cgit

History

#1

Updated by hsbt (Hiroshi SHIBATA) about 1 year ago

  • Related to Misc #10547: How to move the ruby project to git added

Updated by vo.x (Vit Ondruch) about 1 year ago

Wonderful. Thx for working on this.

Have you considered to use Pagure 1 as git fronted instead of cgit?

Updated by normalperson (Eric Wong) about 1 year ago

hsbt@ruby-lang.org wrote:

I prepared git.ruby-lang.org. It's official ruby repository of Git.

Thank you!

  1. To support git repository for bug.ruby-lang.org. PS. redmine could not support remote git repository.

I guess you will need to keep a mirror on the redmine server?
Sorry, I'm not familiar with redmine setup; but I know git :)

Updated by normalperson (Eric Wong) about 1 year ago

v.ondruch@tiscali.cz wrote:

Have you considered to use Pagure [1] as git fronted instead of cgit?

[1]: https://pagure.io/pagure

Pagure would be more a replacement for redmine than a replacement
for cgit, I think. In any case, I prefer loose coupling between
version control and messaging platform.

Updated by Anonymous about 1 year ago

Dne 26.3.2018 v 23:30 Eric Wong napsal(a):

v.ondruch@tiscali.cz wrote:

Have you considered to use Pagure [1] as git fronted instead of cgit?

[1]: https://pagure.io/pagure
Pagure would be more a replacement for redmine than a replacement
for cgit, I think.

It might be of course. It depends which features you enable. But for
example for Fedora sources [1], Pagure is used as git frontend allowing
browsing and accepting PR, managing users, etc, but Bugzilla is used as
a bug tracker. I can imagine similar setup for Ruby, where Redmine is
still used for bug reporting.

Vít

[1] https://src.fedoraproject.org/rpms/ruby
[2] https://bugzilla.redhat.com/

Updated by shevegen (Robert A. Heiler) about 1 year ago

\o/

Hiroshi for president as well!

\o~

~o/

\o/

Updated by Anonymous about 1 year ago

hello,

On 2018-03-26 11:47:38 +0000, hsbt@ruby-lang.org wrote:

Issue #14632 has been reported by hsbt (Hiroshi SHIBATA).


Misc #14632: [ANN] git.ruby-lang.org
https://bugs.ruby-lang.org/issues/14632
[…]
I prepared git.ruby-lang.org. It's official ruby repository of Git.

https://git.ruby-lang.org/ruby.git/

Thanks a lot for this, I'm learning SVN, but still more comfortable
with git.

I had a few issues to retrieve the new repository:

$ git clone git://git.ruby-lang.org/ruby
Cloning into 'ruby'...
load: 0.18 cmd: git 61356 [connec] 4.90r 0.00u 0.01s 0% 3872k
fatal: unable to connect to git.ruby-lang.org:
git.ruby-lang.org[0: 52.192.80.219]: errno=Operation timed out
zsh: exit 128 git clone git://git.ruby-lang.org/ruby

$ nc -v git.ruby-lang.org 9418
nc: connect to git.ruby-lang.org port 9418 (tcp) failed: Operation timed out
zsh: exit 1 nc -v git.ruby-lang.org 9418

I could not connect using git protocol, but maybe it's normal and it
won't be supported? I then attempted HTTPS, but it was a bit slow:

$ time git clone https://git.ruby-lang.org/ruby.git ruby
Cloning into 'ruby'...
load: 0.36 cmd: git-remote-https 61502 [select] 20.97r 0.45u 0.67s 3% 12512k
load: 0.22 cmd: git-remote-https 61502 [select] 51.96r 0.97u 1.41s 2% 15456k
load: 0.96 cmd: git 61525 [running] 60.56r 84.08u 6.33s 100% 178776k
load: 0.96 cmd: git-remote-https 61502 [running] 193.68r 2.17u 2.95s 1% 15996k
load: 0.25 cmd: git-remote-https 61502 [select] 308.69r 6.26u 5.17s 5% 121764k
Checking connectivity: 396310, done.
git clone https://git.ruby-lang.org/ruby.git ruby 108.45s user 14.47s system 26% cpu 7:46.62 total

I'm not complaining, I can of course wait and won't clone it
everyday. But this surprised me a little, am I doing something wrong?
Is it the correct way to fetch this new repository?

Thanks again.

--
Thibault Jouan

Updated by hsbt (Hiroshi SHIBATA) about 1 year ago

I could not connect using git protocol, but maybe it's normal and it
won't be supported? I then attempted HTTPS, but it was a bit slow:

git.r-l.o only supports https protocol now. And I try to improve it clone speed.

Updated by hsbt (Hiroshi SHIBATA) about 1 year ago

Hi all.

I added git.ruby-lang.org to this redmine project.

You can refer git ref like https://bugs.ruby-lang.org/projects/ruby-trunk/repository/ruby-git/revisions/5896936f2ca0c16a68edb557a122125653fa89a7
Redmine expand commit:ruby-git|5896936f2ca0c16a68edb557a122125653fa89a7 to above URL now. (This long syntax is not necessary when git repository was promoted default repository. )

I may change repository identifier (ruby-git ) until completing git migration.

Updated by normalperson (Eric Wong) about 1 year ago

hsbt@ruby-lang.org wrote:

git.r-l.o only supports https protocol now. And I try to
improve it clone speed.

If you haven't already, enable repack.writeBitmaps (and repack)
to speed up the counting objects phase.

Updated by hsbt (Hiroshi SHIBATA) about 1 year ago

If you haven't already, enable repack.writeBitmaps (and repack)

Thanks Eric. I invoked repack and set repack.writeBitmaps to enable on git.r-l.o.

Updated by hsbt (Hiroshi SHIBATA) 10 months ago

Current status is here:

  1. Migrate toolchain like tool/make-snapshot to git.ruby-lang.org from svn.ruby-lang.org.

Not yet, We need to update make-snapshot and backport tool for stable branches.

  1. Improve cgit and access configuration.

Done.

  1. To support git repository for bug.ruby-lang.org. PS. redmine could not support remote git repository.

Done.

  1. Make write-access mode to git.ruby-lang.org. and make the read-only mode to svn.ruby-lang.org.

Not yet.

And We need to configure git-hook for auto-style by svn bot and github sync tool.

Updated by k0kubun (Takashi Kokubun) 5 months ago

To remind us the current status, let me list up the details of remaining tasks for git migration:

Preparations

Operations

  • Make git.ruby-lang.org writable (just announce it), and make svn.ruby-lang.org read-only.
    • Also prohibit force-push to git.ruby-lang.org
  • Uncomment auto-style and update-version in hooks/post-receive.sh

After repository migration to git

  • Use a git commit hash instead of SVN revision in RUBY_REVISION, RUBY_DESCRIPTION
  • Add more git support in:
    • tool/vcs.rb:
      • tool/change_maker.rb
      • tool/file2lastrev.rb
      • tool/make-snapshot (should be ready before 2.7.0-preview1?)
    • Git support to backport to future ruby_2_7 branch:
      • tool/redmine-backporter.rb
      • tool/merger.rb
  • Fix broken git diff on redmine

Caveats

  • Do not create a topic branch on cgit.
  • GitHub repository will still be just a mirror. Don't push "Merge pull request" button.
  • ruby_2_4, ruby_2_5, ruby_2_6 branches will continue to use SVN. Don't push anything to these branches on cgit.

Updated by k0kubun (Takashi Kokubun) 2 months ago

Progress:

I believe there's no more immediate blockers other than "Preparations" and "Operations" written in the above comment. I'll wait hsbt (Hiroshi SHIBATA) for those tasks.

Updated by hsbt (Hiroshi SHIBATA) 2 months ago

k0kubun (Takashi Kokubun) Thanks to your works.

I enabled and disabled some changes on our repository.

Operations

  • Make git.ruby-lang.org writable (just announce it), and make svn.ruby-lang.org read-only.

done. I disabled the writable operation of trunk branch on svn.ruby-lang.org

Also prohibit force-push to git.ruby-lang.org

done. I added denyNonFastForwards = true on git.ruby-lang.org.

Uncomment auto-style and update-version in hooks/post-receive.sh

done. https://github.com/ruby/ruby-commit-hook/commit/0e8fd2abc43e98a703c8bc8d3ec4fe90305409ac

To. the Ruby committers.

Please update your origin variable of git working directory to git@git.ruby-lang.org:ruby.git. If you don't have the writable access, Please share your svn account and the public key of ssh.

I will announce Moving to git from svn later.

Updated by shyouhei (Shyouhei Urabe) 2 months ago

hsbt (Hiroshi SHIBATA) wrote:

done. I disabled the writable operation of trunk branch on svn.ruby-lang.org

Next time tell us BEFORE you disable something, OK?

Updated by hsbt (Hiroshi SHIBATA) 2 months ago

Next time tell us BEFORE you disable something, OK?

I will make an effort next time. Can you join ruby-lang slack team again?
We discuss the internal workflow on this slack.

Updated by shyouhei (Shyouhei Urabe) 2 months ago

hsbt (Hiroshi SHIBATA) wrote:

Next time tell us BEFORE you disable something, OK?

I will make an effort next time. Can you join ruby-lang slack team again?
We discuss the internal workflow on this slack.

I now feel I am excluded from the community because I was not on the chat. Does this mean anyone must join the slack if they have to be considered as a committer?

Updated by naruse (Yui NARUSE) 2 months ago

In this ticket, there's a list of pre and post tasks.
You can always add "pre-announcement" and "post-announcement" in these 3 months.

About git migration, investigation and organizing tasks themselves are also hard work.
I'm very appreciate for people who worked hard for them.

Anyway chat is out of this topic.

Updated by shyouhei (Shyouhei Urabe) 2 months ago

naruse (Yui NARUSE) wrote:

Anyway chat is out of this topic.

Yes please. Don't let the chat involve this kind of things.

Updated by k0kubun (Takashi Kokubun) 2 months ago

Remaining tasks related to Git migration

  • Add more git support in:
    • Git support to backport to future ruby_2_7 branch:
      • tool/merger.rb
  • Discuss the timing to rename the branch "trunk" to "master"? (#15782)

Updated by mame (Yusuke Endoh) 2 months ago

k0kubun (Takashi Kokubun) wrote:

Done.

  • I've created a script named check-email.rb which includes the correspondence table between git ssh key (SVN account name) and committer's email.
  • This script is run on the git server when pushed, and checks if the used ssh key is actually associated with committer's email in each pushed commit.
  • Even if the used key is not listed in the script, you can do push commits until the last of 2019, and the association of ssh key and email is logged. I will occasionally add the logged association to the script manually.
  • In 2020 or later, you will NOT be able to push commits if your email is not listed in the script. Please send PR to github:ruby-commit-hook or contact on hsbt (Hiroshi SHIBATA), naruse (Yui NARUSE), k0kubun (Takashi Kokubun), or me.

Updated by hsbt (Hiroshi SHIBATA) 2 months ago

  • Status changed from Assigned to Closed

I announced https://www.ruby-lang.org/en/news/2019/04/23/move-to-git-from-svn/

We still have some issues. But I will close this.

If you find the issues, Please file them to bugs.ruby-lang.org or https://github.com/ruby/b.r-l.o especially redmine.

Updated by k0kubun (Takashi Kokubun) about 2 months ago

Today and yesterday I updated ruby-commit-hook to deal with race condition and corner cases of notifications, like pushing multiple commits at once. Now ruby-commit-hook should be correctly working for much more situations.

nobu (Nobuyoshi Nakada) and I have fixed tool/*.rb to support Git. I think either of us made Git support for all tool/*.rb which needed changes. Today I changed tool/redmine-backporter.rb and tool/merger.rb to support backporting changes to Git repository for the future 2.7+ stable branches.

I believe we completed Git migration of Ruby repository. Please file another ticket if you find something or a bug in backport tools. Thank you.

#25

Updated by k0kubun (Takashi Kokubun) about 1 month ago

  • Related to Misc #15843: Make "trunk" a symbolic-ref of "master" on git.ruby-lang.org added

Also available in: Atom PDF