You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
instead of many hard copies of the git executable?
This is incorrect. Those are not copies, those are hard links. It is true that Windows Explorer double-counts them, but they do not occupy more disk space than one copy + 127 directory entries pointing to the same start disk sector. You can verify this by running the following command:
$ ls -l /mingw64/bin/git.exe
-rwxr-xr-x 127 johannes 197121 2755352 Dec 15 09:52 /mingw64/bin/git.exe*
See that 127? That's the number of directory entries pointing at the same file contents.
@organds, Any suggestions as to where the best place(s) would be to highlight the information?
It is a question that does come up moderately frequently, so as some one who has been hit by the issue, maybe have a think about where you looked (or even if didn't 'look', why that was) so that we can educate folks and reduce the misunderstandings. Every little helps.
[Would it be worth adding a line to the 'Known Issues' of the Release notes file:///C:/Program%20Files/Git/ReleaseNotes.html, just the head the issue off at the pass]
@orgads yeah, no. I don't think that would be a good idea. First of all, this file would have to be generated only if the files really are hardlinks, and then that file would have to be left out e.g. when copying those files. That's just too fragile.
At some point, we have to bite the bullet and just accept that most users do not really understand their system and need the explanation. I just hope that the number of contributors who relieve the maintainer of explaining this (in favor of chasing much harder bugs, or in favor of trying to hammer down work hours to a normal level) will continue to increase.
@dscho wrote: just accept that most users do not really understand their system and need the explanation (that the "use links in libexec/git-core instead of many hard copies of the git executable" is already done via Windows hardlinks).
Would I be reading this right, that my suggestion of a short(ish) extra line in the 'known issues' section could be that explanation? It may only find 40% of the users but at least its a reduction in complaints... (Should I add it to my list of things to do on retiral;-)
I'm updating the PR (#229 above) that adds this to the Known Issues list (assuming you are commenting about the "127" factor, rather than something I haven't understood)
It seems that the mingw-w64-git package has the hardlinks correct:
$ tar tvf mingw-w64-x86_64-git-2.21.0.1.2481c4cbe9-1-any.pkg.tar.xz
|grep 'git\.exe'
-rwxr-xr-x ContainerAdministrator/380929 40728 2019-02-26 20:48 cmd/git.exe
hrwxr-xr-x ContainerAdministrator/380929 0 2019-02-26 20:48 mingw64/bin/git.exe link to mingw64/bin/git-receive-pack.exe
hrwxr-xr-x ContainerAdministrator/380929 0 2019-02-26 20:48 mingw64/libexec/git-core/git.exe link to mingw64/bin/git-receive-pack.exe
Meaning: something must be going wrong when the installer does its magic. It does call HardlinkOrCopyGit(), or at least is supposed to, for all builtins here.
So the builtins are determined from the file /mingw64/share/git/builtins.txt, read here. And that file does not contain /mingw64/libexec/git-core/git.exe here.
That builtins.txt file is generated during installer/release.sh <version> time, and I think it probably determines which files to include by looking which files were replaced by the git-wrapper.exe in git-extra's post-install script.
You see, we got so many concerns about that "massive" filesystem usage that we started to ship at least our portable Git with the builtins replaced not by hardlinked versions of git.exe, but by the Git wrapper (which basically just re-spawns the real git.exe with the builtin's command name inserted as first parameter). And that is why git-extra's post-install script edits the files installed by mingw-w64-git.
Now, the reason why it does not include /mingw64/libexec/git-core/git.exe is obvious: that is not a builtin, if we replaced that by the Git wrapper, it would either cause an infinite loop, or it would not find the real git.exe to execute.
I guess the best idea would be to teach git-wrapper.c to detect when it was called as .../libexec/git-core/git.exe and spawn .../bin/git.exe in that case, much like /cmd/git.exe does.
To extend this to /mingw64/libexec/git-core/git.exe, we would have to use 3 instead.
And then we could teach the print-builtins target in the mingw-w64-git.mak file to also print that path.
Of course, this all would need to be tested by
patching MINGW-packages,
building and installing that mingw-w64-x86_64-git package
re-installing git-extra (to replace the builtins with the Git wrapper)
verifying that /mingw64/libexec/git-core/git.exe works as intended.
Seeing as I spent the last 38 minutes investigating it this far, I now think I should be doing more pressing things, and leave the rest to interested parties (I am looking at you, @orgads 😄 )
Activity
dscho commentedon Dec 23, 2018
Only if you build Git yourself.
However:
This is incorrect. Those are not copies, those are hard links. It is true that Windows Explorer double-counts them, but they do not occupy more disk space than one copy + 127 directory entries pointing to the same start disk sector. You can verify this by running the following command:
$ ls -l /mingw64/bin/git.exe -rwxr-xr-x 127 johannes 197121 2755352 Dec 15 09:52 /mingw64/bin/git.exe*See that 127? That's the number of directory entries pointing at the same file contents.
PhilipOakley commentedon Dec 23, 2018
@organds, Any suggestions as to where the best place(s) would be to highlight the information?
It is a question that does come up moderately frequently, so as some one who has been hit by the issue, maybe have a think about where you looked (or even if didn't 'look', why that was) so that we can educate folks and reduce the misunderstandings. Every little helps.
[Would it be worth adding a line to the 'Known Issues' of the Release notes file:///C:/Program%20Files/Git/ReleaseNotes.html, just the head the issue off at the pass]
orgads commentedon Dec 24, 2018
I see. So you have one extra executable: libexec/git-core/git.exe:
@PhilipOakley : Maybe add an empty file inside libexec/git-core named
___MOST-EXECUTABLES-ARE-HARDLINKS___. That would have attracted my attention :)dscho commentedon Dec 26, 2018
@orgads yeah, no. I don't think that would be a good idea. First of all, this file would have to be generated only if the files really are hardlinks, and then that file would have to be left out e.g. when copying those files. That's just too fragile.
At some point, we have to bite the bullet and just accept that most users do not really understand their system and need the explanation. I just hope that the number of contributors who relieve the maintainer of explaining this (in favor of chasing much harder bugs, or in favor of trying to hammer down work hours to a normal level) will continue to increase.
PhilipOakley commentedon Jan 3, 2019
Would I be reading this right, that my suggestion of a short(ish) extra line in the 'known issues' section could be that explanation? It may only find 40% of the users but at least its a reduction in complaints... (Should I add it to my list of things to do on retiral;-)
dscho commentedon Jan 7, 2019
@PhilipOakley sounds good ;-)
ReleaseNotes: duplicated files are hard links
dscho commentedon Feb 27, 2019
Addressed via git-for-windows/build-extra#229
orgads commentedon Mar 1, 2019
What about this issue?
PhilipOakley commentedon Mar 1, 2019
I'm updating the PR (#229 above) that adds this to the Known Issues list (assuming you are commenting about the "127" factor, rather than something I haven't understood)
orgads commentedon Mar 1, 2019
I don't refer to the factor. I refer to the additional executable libexec/git-core/git.exe, which is not a hard-link.
dscho commentedon Mar 1, 2019
That's a bug, and if you can figure out a fix, I'll be happy to accept a PR.
dscho commentedon Mar 1, 2019
(To clarify: this extra
git.exeshould also be a hardlink.)orgads commentedon Mar 1, 2019
First observation: make install creates all as hard links, so it's probably something in the installer...
dscho commentedon Mar 1, 2019
It seems that the
mingw-w64-gitpackage has the hardlinks correct:Meaning: something must be going wrong when the installer does its magic. It does call
HardlinkOrCopyGit(), or at least is supposed to, for all builtins here.dscho commentedon Mar 1, 2019
So the builtins are determined from the file
/mingw64/share/git/builtins.txt, read here. And that file does not contain/mingw64/libexec/git-core/git.exehere.dscho commentedon Mar 1, 2019
That
builtins.txtfile is generated duringinstaller/release.sh <version>time, and I think it probably determines which files to include by looking which files were replaced by thegit-wrapper.exeingit-extra's post-install script.You see, we got so many concerns about that "massive" filesystem usage that we started to ship at least our portable Git with the builtins replaced not by hardlinked versions of
git.exe, but by the Git wrapper (which basically just re-spawns the realgit.exewith the builtin's command name inserted as first parameter). And that is whygit-extra's post-install script edits the files installed bymingw-w64-git.dscho commentedon Mar 1, 2019
Oh, and I was wrong about
installer/release.shgenerating thatbuiltinsfile. It is https://github.com/git-for-windows/MINGW-packages/blob/01d3f0e810b7e5a37488d97c860ba3df20685412/mingw-w64-git/PKGBUILD#L154Now, the reason why it does not include
/mingw64/libexec/git-core/git.exeis obvious: that is not a builtin, if we replaced that by the Git wrapper, it would either cause an infinite loop, or it would not find the realgit.exeto execute.I guess the best idea would be to teach
git-wrapper.cto detect when it was called as.../libexec/git-core/git.exeand spawn.../bin/git.exein that case, much like/cmd/git.exedoes.Now, https://github.com/git-for-windows/MINGW-packages/blob/01d3f0e810b7e5a37488d97c860ba3df20685412/mingw-w64-git/git-wrapper.c#L696 tries to be very careful to strip only one directory level (the
cmdpart ofC:\Program Files\Git\cmd\git.exe) in order to look whether we are in an MSYS2-based setup (essentially by looking whether<top-level-directory>\mingw64\binexists).To extend this to
/mingw64/libexec/git-core/git.exe, we would have to use 3 instead.And then we could teach the
print-builtinstarget in themingw-w64-git.makfile to also print that path.Of course, this all would need to be tested by
mingw-w64-x86_64-gitpackagegit-extra(to replace the builtins with the Git wrapper)/mingw64/libexec/git-core/git.exeworks as intended.Seeing as I spent the last 38 minutes investigating it this far, I now think I should be doing more pressing things, and leave the rest to interested parties (I am looking at you, @orgads 😄 )