Why is my git-http-backend solution using WebDAV on push?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Why is my git-http-backend solution using WebDAV on push?

Luke Madhanga
I've implemented a PHP wrapper for git http backend which works well.
I've done this to give me advanced control of who has access to
repositories on my server. You can see the implementation on
http://stackoverflow.com/questions/36998492/channel-git-on-the-server-calls-through-php/37242591#37242591.
I can pull from the server okay and all works well. However, I cannot
push. When I read my trace code to see where it fails, I see that the
last request is a PROPFIND request. The URL for this request does not
have any of the usual 'info/refs' etc. that one usually gets on git
calls.

During reading of the documentation and reviewing how git-http-backend
should be implemented, I can see that it should not be making a WebDAV
request. I have also come across posts where people implementing
http-backend the 'normal' way have also had this problem, e.g.
http://serverfault.com/questions/390864/git-push-over-http-using-git-http-backend-and-apache-is-not-working



So the question is, why is WebDAV being used on push?




Trace code from the push:

[2016-05-25 09:49:35] ==========================
[2016-05-25 09:49:35] REQUEST: {
    "q": "p\/git-backend\/run\/1\/info\/refs",
    "service": "git-receive-pack"
}
[2016-05-25 09:49:35] SERVER: {
    "REDIRECT_STATUS": "200",
    "HTTP_HOST": "...",
    "HTTP_USER_AGENT": "git\/2.7.4",
    "HTTP_ACCEPT": "*\/*",
    "HTTP_ACCEPT_ENCODING": "gzip",
    "HTTP_ACCEPT_LANGUAGE": "en-GB, en;q=0.9, *;q=0.8",
    "HTTP_PRAGMA": "no-cache",
    "PATH": "...",
    "SERVER_SIGNATURE": "<address>Apache\/2.4.18 (Ubuntu) Server at
... Port 80<\/address>\n",
    "SERVER_SOFTWARE": "Apache\/2.4.18 (Ubuntu)",
    "SERVER_NAME": "...",
    "SERVER_ADDR": "...",
    "SERVER_PORT": "80",
    "REMOTE_ADDR": "...",
    "DOCUMENT_ROOT": "...",
    "REQUEST_SCHEME": "http",
    "CONTEXT_PREFIX": "",
    "CONTEXT_DOCUMENT_ROOT": "...",
    "SERVER_ADMIN": "...",
    "SCRIPT_FILENAME": "...",
    "REMOTE_PORT": "49630",
    "REDIRECT_URL": "\/p\/git-backend\/run\/1\/info\/refs",
    "REDIRECT_QUERY_STRING":
"q=p\/git-backend\/run\/1\/info\/refs&service=git-receive-pack",
    "GATEWAY_INTERFACE": "CGI\/1.1",
    "SERVER_PROTOCOL": "HTTP\/1.1",
    "REQUEST_METHOD": "GET",
    "QUERY_STRING":
"q=p\/git-backend\/run\/1\/info\/refs&service=git-receive-pack",
    "REQUEST_URI":
"\/p\/git-backend\/run\/1\/info\/refs?service=git-receive-pack",
    "SCRIPT_NAME": "\/index.php",
    "PHP_SELF": "\/index.php",
    "REQUEST_TIME_FLOAT": 1464162575.091,
    "REQUEST_TIME": 1464162575
}
[2016-05-25 09:49:35] Path: /core.git/info/refs?service=git-receive-pack
[2016-05-25 09:49:35] Cleaned, result only:
[2016-05-25 09:49:35] f4648182f5f8eee082c37a83a0072cfc4210e5c5 refs/heads/master
8c4efcd77809bc9b94a59cf94653add8007c6b7d refs/heads/zztest
[2016-05-25 09:49:35] ==========================
[2016-05-25 09:49:35] REQUEST: {
    "q": "p\/git-backend\/run\/1\/HEAD"
}
[2016-05-25 09:49:35] SERVER: {
    "REDIRECT_STATUS": "200",
    "HTTP_HOST": "...",
    "HTTP_USER_AGENT": "git\/2.7.4",
    "HTTP_ACCEPT": "*\/*",
    "HTTP_ACCEPT_ENCODING": "gzip",
    "HTTP_ACCEPT_LANGUAGE": "en-GB, en;q=0.9, *;q=0.8",
    "HTTP_PRAGMA": "no-cache",
    "PATH": "...",
    "SERVER_SIGNATURE": "<address>Apache\/2.4.18 (Ubuntu) Server at
... Port 80<\/address>\n",
    "SERVER_SOFTWARE": "Apache\/2.4.18 (Ubuntu)",
    "SERVER_NAME": "...",
    "SERVER_ADDR": "...",
    "SERVER_PORT": "80",
    "REMOTE_ADDR": "...",
    "DOCUMENT_ROOT": "...",
    "REQUEST_SCHEME": "http",
    "CONTEXT_PREFIX": "",
    "CONTEXT_DOCUMENT_ROOT": "...",
    "SERVER_ADMIN": "...",
    "SCRIPT_FILENAME": "...",
    "REMOTE_PORT": "49630",
    "REDIRECT_URL": "\/p\/git-backend\/run\/1\/HEAD",
    "REDIRECT_QUERY_STRING": "q=p\/git-backend\/run\/1\/HEAD",
    "GATEWAY_INTERFACE": "CGI\/1.1",
    "SERVER_PROTOCOL": "HTTP\/1.1",
    "REQUEST_METHOD": "GET",
    "QUERY_STRING": "q=p\/git-backend\/run\/1\/HEAD",
    "REQUEST_URI": "\/p\/git-backend\/run\/1\/HEAD",
    "SCRIPT_NAME": "\/index.php",
    "PHP_SELF": "\/index.php",
    "REQUEST_TIME_FLOAT": 1464162575.266,
    "REQUEST_TIME": 1464162575
}
[2016-05-25 09:49:35] Path: /core.git/HEAD
[2016-05-25 09:49:35] Cleaned, result only:
[2016-05-25 09:49:35] ref: refs/heads/master
[2016-05-25 09:49:35] ==========================
[2016-05-25 09:49:35] REQUEST: {
    "q": "p\/git-backend\/run\/1\/"
}
[2016-05-25 09:49:35] SERVER: {
    "REDIRECT_STATUS": "200",
    "HTTP_HOST": "...",
    "HTTP_USER_AGENT": "git\/2.7.4",
    "HTTP_ACCEPT": "*\/*",
    "HTTP_DEPTH": "0",
    "CONTENT_TYPE": "text\/xml",
    "CONTENT_LENGTH": "167",
    "HTTP_EXPECT": "100-continue",
    "PATH": "...",
    "SERVER_SIGNATURE": "<address>Apache\/2.4.18 (Ubuntu) Server at
... Port 80<\/address>\n",
    "SERVER_SOFTWARE": "Apache\/2.4.18 (Ubuntu)",
    "SERVER_NAME": "...",
    "SERVER_ADDR": "...",
    "SERVER_PORT": "80",
    "REMOTE_ADDR": "...",
    "DOCUMENT_ROOT": "...",
    "REQUEST_SCHEME": "http",
    "CONTEXT_PREFIX": "",
    "CONTEXT_DOCUMENT_ROOT": "...",
    "SERVER_ADMIN": "...",
    "SCRIPT_FILENAME": "...",
    "REMOTE_PORT": "49632",
    "REDIRECT_URL": "\/p\/git-backend\/run\/1\/",
    "REDIRECT_QUERY_STRING": "q=p\/git-backend\/run\/1\/",
    "GATEWAY_INTERFACE": "CGI\/1.1",
    "SERVER_PROTOCOL": "HTTP\/1.1",
    "REQUEST_METHOD": "PROPFIND",
    "QUERY_STRING": "q=p\/git-backend\/run\/1\/",
    "REQUEST_URI": "\/p\/git-backend\/run\/1\/",
    "SCRIPT_NAME": "\/index.php",
    "PHP_SELF": "\/index.php",
    "REQUEST_TIME_FLOAT": 1464162575.711,
    "REQUEST_TIME": 1464162575
}
[2016-05-25 09:49:35] Path: /core.git/
[2016-05-25 09:49:35] <?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:prop xmlns:R="http://.../p/git-backend/run/1/">
<D:supportedlock/>
</D:prop>
</D:propfind>
[2016-05-25 09:49:35] Exception: Request not supported: '/path/to/git/repo.git/'

When run using curl verbose, the output is


* Couldn't find host xxx in the .netrc file; using defaults
*   Trying xxx...
* Connected to xxx (xxx) port 80 (#0)
> GET /p/git-backend/run/1/info/refs?service=git-receive-pack HTTP/1.1
Host: xxx
User-Agent: git/2.7.4
Accept: */*
Accept-Encoding: gzip
Accept-Language: en-GB, en;q=0.9, *;q=0.8
Pragma: no-cache

< HTTP/1.1 200 OK
< Date: Wed, 25 May 2016 19:00:25 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Set-Cookie: PHPSESSID=yyy; path=/
< Expires: Fri, 01 Jan 1980 00:00:00 GMT
< Cache-Control: no-cache, max-age=0, must-revalidate
< Pragma: no-cache
< Vary: Accept-Encoding
< Content-Encoding: gzip
< Content-Length: 109
< Content-Type: text/plain;charset=UTF-8
<
* Connection #0 to host madhan.ga left intact
* Couldn't find host xxx in the .netrc file; using defaults
* Found bundle for host xxx: 0x9bade0 [can pipeline]
* Re-using existing connection! (#0) with host xxx
* Connected to xxx (xxx) port 80 (#0)
> GET /p/git-backend/run/1/HEAD HTTP/1.1
Host: xxx
User-Agent: git/2.7.4
Accept: */*
Accept-Encoding: gzip
Accept-Language: en-GB, en;q=0.9, *;q=0.8
Pragma: no-cache

< HTTP/1.1 200 OK
< Date: Wed, 25 May 2016 19:00:25 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Set-Cookie: PHPSESSID=zzz; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate
< Pragma: no-cache
< Content-Length: 23
< Content-Type: text/plain;charset=UTF-8
<
* Connection #0 to host xxx left intact
* Couldn't find host xxx in the .netrc file; using defaults
*   Trying xxx...
* Connected to xxx (xxx) port 80 (#0)
> PROPFIND /p/git-backend/run/1/ HTTP/1.1
Host: xxx
User-Agent: git/2.7.4
Accept: */*
Depth: 0
Content-Type: text/xml
Content-Length: 167
Expect: 100-continue

< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* HTTP 1.0, assume close after body
< HTTP/1.0 100 Continue
< Date: Wed, 25 May 2016 19:00:25 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Set-Cookie: PHPSESSID=aaa; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate
< Pragma: no-cache
< Content-Length: 0
< Connection: close
< Content-Type: text/json; charset=utf-8
* Empty reply from server
* Closing connection 0
error: Cannot access URL http://xxx/p/git-backend/run/1/, return code 52
fatal: git-http-push failed
error: failed to push some refs to 'http://xxx/p/git-backend/run/1'



--
Yours,
Luke Madhanga
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: Why is my git-http-backend solution using WebDAV on push?

Jeff King
On Wed, May 25, 2016 at 09:28:21PM +0100, Luke Madhanga wrote:

> I've implemented a PHP wrapper for git http backend which works well.
> I've done this to give me advanced control of who has access to
> repositories on my server. You can see the implementation on
> http://stackoverflow.com/questions/36998492/channel-git-on-the-server-calls-through-php/37242591#37242591.
> I can pull from the server okay and all works well. However, I cannot
> push. When I read my trace code to see where it fails, I see that the
> last request is a PROPFIND request. The URL for this request does not
> have any of the usual 'info/refs' etc. that one usually gets on git
> calls.

If the git client detects that the server doesn't implement the
smart-http protocol, it will automatically downgrade to the older,
"dumb" protocol which is less efficient.

For fetching/cloning, this might make your test appear to work, even
though it is downgrading behind the scenes. You should be able to check
GIT_CURL_VERBOSE output to tell the difference; the smart protocol will
POST to .../git-upload-pack, whereas the dumb protocol will download the
individual packfiles and objects directly.

For pushing, the dumb protocol uses WebDAV, which is what you're seeing.

So the question is: why doesn't the client think your server is a smart
server?

Just skimming your output, I'd guess:

> > GET /p/git-backend/run/1/info/refs?service=git-receive-pack HTTP/1.1
> Host: xxx
> User-Agent: git/2.7.4
> Accept: */*
> Accept-Encoding: gzip
> Accept-Language: en-GB, en;q=0.9, *;q=0.8
> Pragma: no-cache
>
> < HTTP/1.1 200 OK
> < Date: Wed, 25 May 2016 19:00:25 GMT
> < Server: Apache/2.4.18 (Ubuntu)
> < Set-Cookie: PHPSESSID=yyy; path=/
> < Expires: Fri, 01 Jan 1980 00:00:00 GMT
> < Cache-Control: no-cache, max-age=0, must-revalidate
> < Pragma: no-cache
> < Vary: Accept-Encoding
> < Content-Encoding: gzip
> < Content-Length: 109
> < Content-Type: text/plain;charset=UTF-8
> <

The content-type here should be:

  application/x-git-receive-pack-advertisement

The body content should also include "# service=git-receive-pack" on the
first line, but if you are successfully calling into git-http-backend, I
think it should take care of that detail.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: Why is my git-http-backend solution using WebDAV on push?

Luke Madhanga
Hmmmm. Interesting.

When you look at the PHP code, you'll see the following

    $res = self::proc_open("{$gitcoredir}/git-http-backend", [],
$gitdir, true, [...]);
    ...
    $resbits = explode("\n", $res);
    foreach ($resbits as $index => $header) {
        if ($header && strpos($header, ':') !== false) {
            // Headers
            header($header);
            unset($resbits[$index]);
        } else {
            // First blank line is the space between the headers and
the start of the response from Git
            break;
        }
    }
    echo ltrim(implode("\n", $resbits));
    exit;


Everything being returned is from a direct call to the git-http-backend.

A manual CLI call to git-http-backend doesn't include
'application/x-git-receive-pack-advertisement'

REQUEST_METHOD=GET GIT_PROJECT_ROOT=/path/to/core/
PATH_INFO=/repo.git/info/refs /usr/lib/git-core/git-http-backend

The above command outputs

Expires: Fri, 01 Jan 1980 00:00:00 GMT
Pragma: no-cache
Cache-Control: no-cache, max-age=0, must-revalidate
Content-Length: 118
Content-Type: text/plain

f4648182f5f8eee082c37a83a0072cfc4210e5c5 refs/heads/master
8c4efcd77809bc9b94a59cf94653add8007c6b7d refs/heads/zztest
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: Why is my git-http-backend solution using WebDAV on push?

Luke Madhanga
Thanks for the response btw

On 25 May 2016 at 22:54, Luke Madhanga <[hidden email]> wrote:

> Hmmmm. Interesting.
>
> When you look at the PHP code, you'll see the following
>
>     $res = self::proc_open("{$gitcoredir}/git-http-backend", [],
> $gitdir, true, [...]);
>     ...
>     $resbits = explode("\n", $res);
>     foreach ($resbits as $index => $header) {
>         if ($header && strpos($header, ':') !== false) {
>             // Headers
>             header($header);
>             unset($resbits[$index]);
>         } else {
>             // First blank line is the space between the headers and
> the start of the response from Git
>             break;
>         }
>     }
>     echo ltrim(implode("\n", $resbits));
>     exit;
>
>
> Everything being returned is from a direct call to the git-http-backend.
>
> A manual CLI call to git-http-backend doesn't include
> 'application/x-git-receive-pack-advertisement'
>
> REQUEST_METHOD=GET GIT_PROJECT_ROOT=/path/to/core/
> PATH_INFO=/repo.git/info/refs /usr/lib/git-core/git-http-backend
>
> The above command outputs
>
> Expires: Fri, 01 Jan 1980 00:00:00 GMT
> Pragma: no-cache
> Cache-Control: no-cache, max-age=0, must-revalidate
> Content-Length: 118
> Content-Type: text/plain
>
> f4648182f5f8eee082c37a83a0072cfc4210e5c5 refs/heads/master
> 8c4efcd77809bc9b94a59cf94653add8007c6b7d refs/heads/zztest



--
Yours,
Luke Madhanga
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: Why is my git-http-backend solution using WebDAV on push?

Junio C Hamano
Luke Madhanga <[hidden email]> writes:

>> A manual CLI call to git-http-backend doesn't include
>> 'application/x-git-receive-pack-advertisement'
>>
>> REQUEST_METHOD=GET GIT_PROJECT_ROOT=/path/to/core/
>> PATH_INFO=/repo.git/info/refs /usr/lib/git-core/git-http-backend

The request client makes to probe is (taking it from Peff's message
that is quoting from your trace):

> > GET /p/git-backend/run/1/info/refs?service=git-receive-pack HTTP/1.1

Your manual CLI call seems not to have "?service=git-receive-pack"
anywhere.  Where did it go?  QUERY_STRING, perhaps?

Here is what I am observing:

    $ GIT_HTTP_EXPORT_ALL=Yes \
    > REQUEST_METHOD=GET \
    > GIT_PROJECT_ROOT=$(pwd)/.git \
    > PATH_INFO='/info/refs' \
    > QUERY_STRING=service=git-receive-pack \
    > git -c http.receivepack=yes http-backend 2>&1 | sed -e '/^.$/q'
    Expires: Fri, 01 Jan 1980 00:00:00 GMT
    Pragma: no-cache
    Cache-Control: no-cache, max-age=0, must-revalidate
    Content-Type: application/x-git-receive-pack-advertisement


>>
>> The above command outputs
>>
>> Expires: Fri, 01 Jan 1980 00:00:00 GMT
>> Pragma: no-cache
>> Cache-Control: no-cache, max-age=0, must-revalidate
>> Content-Length: 118
>> Content-Type: text/plain
>>
>> f4648182f5f8eee082c37a83a0072cfc4210e5c5 refs/heads/master
>> 8c4efcd77809bc9b94a59cf94653add8007c6b7d refs/heads/zztest
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html