Access different NAMESPACE of remote repo from client side

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Access different NAMESPACE of remote repo from client side

Jiang Xin
GIT_NAMESPACE is designed to be used mainly on the server side, that
the server can serve multiple git repositories while share one single
repository storage using different GIT_NAMESPACE settings.

Since we know that one remote repository hosts multiple namespaces,
can we handle different namespaces in one local repository? Or can
we access the proper namespace of the remote repository without
complicated server settings?

At least there are three solutions for ssh protocol: pass namespace
through environment, pass namespace in URL, or pass namespace from
the proper settings of remote.<name>.receivepack and
remote.<name>.uploadpack.

Solution 1: passing the namespace through environment.

 1. Set '/etc/sshd_config' in the server side as the following,
    so that the ssh server can accept GIT_NAMESPACE environment.

        AcceptEnv LANG LC_* GIT_NAMESPACE

 2. In the client side, When connect to ssh server, must send the
    GIT_NAMESPACE environment. This can be done with a remote-ext
    url:

        $ git remote add foo \
          'ext::ssh -o SendEnv=GIT_NAMESPACE git@server %S 'path/to/repo.git'

Then the remote "foo" is GIT_NAMESPACE aware, but when operate on
this remote, must provide proper "--namespace" option.

    $ git --namespace=foo push foo master
    $ git --namespace=foo fetch foo
    $ git --namespace=foo ls-remote foo
    $ git --namespace=foo remote prune foo
    $ git --namespace=foo archive --remote foo HEAD

But provide a "--namespace" option is error-prone, but we may invent
"remote.<name>.namespace" or something to set GIT_NAMESPACE
automatically when push to or fetch from remote server.

Solution 2: passing the namespace in URL.

Again use remote-ext style URL to access remote repository:

    $ git remote add foo \
        'ext::ssh git@server git --namespace foo %s path/to/repo.git'

    $ git remote add bar \
        'ext::ssh git@server git --namespace bar %s path/to/repo.git'

But if the remote server use a limit shell (such as git-shell or
gitolite), the above URLs won't work. This is because these git
specific shell (git-shell or gitolite) do not like options.

Solution 3: use custom receivepack and uploadpack. e.g.

    [remote "foo"]
        url = ssh://git@server/path/to/repo.git
        receivepack = git --namespace foo receive-pack
        uploadpack = git --namespace foo upload-pack
        fetch = +refs/heads/*:refs/remotes/foo/*

    [remote "bar"]
        url = ssh://git@server/path/to/repo.git
        receivepack = git --namespace bar receive-pack
        uploadpack = git --namespace bar upload-pack
        fetch = +refs/heads/*:refs/remotes/foo/*

Just like solution 2, these settings won't work without a patched
git-shell or gitolite.

I will send a patch in next email to address the latter two cases.


--
Jiang Xin
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

[PATCH] shell: allow git command with --namespace option

Jiang Xin
GIT_NAMESPACE is designed to be used mainly on the server side, that
the server can serve multiple git repositories while share one single
repository storage using different GIT_NAMESPACE settings.

Since we know that one remote repository hosts multiple namespaces,
can we handle different namespaces in one local repository? Or can
we access the proper namespace of the remote repository without
complicated server settings?

We can access namespace of remote repository using a proper formated
URL or with the help of settings of "remote.<name>.receivepack" and
"remote.<name>.uploadpack". E.g.

Use remote-ext style URL to access specific namespace of the remote:

    $ git remote add foo \
        'ext::ssh git@server git --namespace foo %s path/to/repo.git'

    $ git remote add bar \
        'ext::ssh git@server git --namespace bar %s path/to/repo.git'

Or set "remote.<name>.receivepack" and "remote.<name>.uploadpack".

    [remote "foo"]
        url = ssh://git@server/path/to/repo.git
        receivepack = git --namespace foo receive-pack
        uploadpack = git --namespace foo upload-pack
        fetch = +refs/heads/*:refs/remotes/foo/*

    [remote "bar"]
        url = ssh://git@server/path/to/repo.git
        receivepack = git --namespace bar receive-pack
        uploadpack = git --namespace bar upload-pack
        fetch = +refs/heads/*:refs/remotes/foo/*

But if the remote ssh server use a limit shell (such as git-shell
or gitolite), the above URLs won't work. This is because these git
specific shell (git-shell or gitolite) do not like options.

This patch makes git-shell aware of the "--namespace" option.

Signed-off-by: Jiang Xin <[hidden email]>
---
 shell.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 46 insertions(+), 1 deletion(-)

diff --git a/shell.c b/shell.c
index 66350b2..a619577 100644
--- a/shell.c
+++ b/shell.c
@@ -25,6 +25,50 @@ static int do_generic_cmd(const char *me, char *arg)
  return execv_git_cmd(my_argv);
 }
 
+static int do_git_with_opts_cmd(const char *me, char *arg)
+{
+ const char *allowed_cmds[] = {
+ "receive-pack",
+ "upload-pack",
+ "upload-archive",
+ NULL,
+ };
+ const char **user_argv, **p;
+ const char *cmd = NULL;
+ int count;
+
+ count = split_cmdline(arg, &user_argv);
+ if (count < 0) {
+ die ("Invalid command format '%s'\n", arg);
+ }
+
+ if (count == 3 && !strncmp("--namespace=", *user_argv, 12)) {
+ cmd = user_argv[1];
+ p = user_argv + 2;
+ } else if (count == 4 && !strcmp("--namespace", *user_argv)) {
+ cmd = user_argv[2];
+ p = user_argv + 3;
+ } else {
+ cmd = user_argv[0];
+ p = user_argv + 1;
+ }
+
+ if (cmd) {
+ /* last arg is path of repository */
+ if (!*p || *(p+1))
+ die("bad argument");
+
+ for (p = allowed_cmds; *p; p++) {
+ if (strcmp(*p, cmd))
+ continue;
+ setup_path();
+ return execv_git_cmd(user_argv);
+ }
+ }
+
+ die("bad command");
+}
+
 static int do_cvs_cmd(const char *me, char *arg)
 {
  const char *cvsserver_argv[3] = {
@@ -138,6 +182,7 @@ static struct commands {
  { "git-receive-pack", do_generic_cmd },
  { "git-upload-pack", do_generic_cmd },
  { "git-upload-archive", do_generic_cmd },
+ { "git", do_git_with_opts_cmd },
  { "cvs", do_cvs_cmd },
  { NULL },
 };
@@ -185,7 +230,7 @@ int main(int argc, char **argv)
  }
 
  prog = xstrdup(argv[2]);
- if (!strncmp(prog, "git", 3) && isspace(prog[3]))
+ if (!strncmp(prog, "git", 3) && isspace(prog[3]) && isalpha(prog[4]))
  /* Accept "git foo" as if the caller said "git-foo". */
  prog[3] = '-';
 
--
1.8.5.rc2.2.g0469850

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: Access different NAMESPACE of remote repo from client side

Sitaram Chamarty
In reply to this post by Jiang Xin
On 11/15/2013 01:49 PM, Jiang Xin wrote:

> GIT_NAMESPACE is designed to be used mainly on the server side, that
> the server can serve multiple git repositories while share one single
> repository storage using different GIT_NAMESPACE settings.
>
> Since we know that one remote repository hosts multiple namespaces,
> can we handle different namespaces in one local repository? Or can
> we access the proper namespace of the remote repository without
> complicated server settings?
>
> At least there are three solutions for ssh protocol: pass namespace
> through environment, pass namespace in URL, or pass namespace from
> the proper settings of remote.<name>.receivepack and
> remote.<name>.uploadpack.
>
> Solution 1: passing the namespace through environment.
>
>  1. Set '/etc/sshd_config' in the server side as the following,
>     so that the ssh server can accept GIT_NAMESPACE environment.
>
>         AcceptEnv LANG LC_* GIT_NAMESPACE
>
>  2. In the client side, When connect to ssh server, must send the
>     GIT_NAMESPACE environment. This can be done with a remote-ext
>     url:
>
>         $ git remote add foo \
>           'ext::ssh -o SendEnv=GIT_NAMESPACE git@server %S 'path/to/repo.git'
>
> Then the remote "foo" is GIT_NAMESPACE aware, but when operate on
> this remote, must provide proper "--namespace" option.
>
>     $ git --namespace=foo push foo master
>     $ git --namespace=foo fetch foo
>     $ git --namespace=foo ls-remote foo
>     $ git --namespace=foo remote prune foo
>     $ git --namespace=foo archive --remote foo HEAD
>
> But provide a "--namespace" option is error-prone, but we may invent
> "remote.<name>.namespace" or something to set GIT_NAMESPACE
> automatically when push to or fetch from remote server.
>
> Solution 2: passing the namespace in URL.
>
> Again use remote-ext style URL to access remote repository:
>
>     $ git remote add foo \
>         'ext::ssh git@server git --namespace foo %s path/to/repo.git'
>
>     $ git remote add bar \
>         'ext::ssh git@server git --namespace bar %s path/to/repo.git'
>
> But if the remote server use a limit shell (such as git-shell or
> gitolite), the above URLs won't work. This is because these git
> specific shell (git-shell or gitolite) do not like options.
>
> Solution 3: use custom receivepack and uploadpack. e.g.
>
>     [remote "foo"]
>         url = ssh://git@server/path/to/repo.git
>         receivepack = git --namespace foo receive-pack
>         uploadpack = git --namespace foo upload-pack
>         fetch = +refs/heads/*:refs/remotes/foo/*
>
>     [remote "bar"]
>         url = ssh://git@server/path/to/repo.git
>         receivepack = git --namespace bar receive-pack
>         uploadpack = git --namespace bar upload-pack
>         fetch = +refs/heads/*:refs/remotes/foo/*
>
> Just like solution 2, these settings won't work without a patched
> git-shell or gitolite.

Gitolite has a namespaces branch that handles namespaces as described in
http://gitolite.com/gitolite/namespaces.html

Briefly, it recognises that you can have a "main" repo off of which
several developer might want to hang their logical repos.

It also recognises that the actual names of the logical repos will
follow some pattern that may include the name of the developer also, and
provides a way to derive the name of the physical repo from the logical
one.

There is an example or two in that link.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: Access different NAMESPACE of remote repo from client side

Sitaram Chamarty
On 11/15/2013 07:55 PM, Sitaram Chamarty wrote:

> On 11/15/2013 01:49 PM, Jiang Xin wrote:
>> GIT_NAMESPACE is designed to be used mainly on the server side, that
>> the server can serve multiple git repositories while share one single
>> repository storage using different GIT_NAMESPACE settings.
>>
>> Since we know that one remote repository hosts multiple namespaces,
>> can we handle different namespaces in one local repository? Or can
>> we access the proper namespace of the remote repository without
>> complicated server settings?
>>
>> At least there are three solutions for ssh protocol: pass namespace
>> through environment, pass namespace in URL, or pass namespace from
>> the proper settings of remote.<name>.receivepack and
>> remote.<name>.uploadpack.
>>
>> Solution 1: passing the namespace through environment.
>>
>>  1. Set '/etc/sshd_config' in the server side as the following,
>>     so that the ssh server can accept GIT_NAMESPACE environment.
>>
>>         AcceptEnv LANG LC_* GIT_NAMESPACE
>>
>>  2. In the client side, When connect to ssh server, must send the
>>     GIT_NAMESPACE environment. This can be done with a remote-ext
>>     url:
>>
>>         $ git remote add foo \
>>           'ext::ssh -o SendEnv=GIT_NAMESPACE git@server %S 'path/to/repo.git'
>>
>> Then the remote "foo" is GIT_NAMESPACE aware, but when operate on
>> this remote, must provide proper "--namespace" option.
>>
>>     $ git --namespace=foo push foo master
>>     $ git --namespace=foo fetch foo
>>     $ git --namespace=foo ls-remote foo
>>     $ git --namespace=foo remote prune foo
>>     $ git --namespace=foo archive --remote foo HEAD
>>
>> But provide a "--namespace" option is error-prone, but we may invent
>> "remote.<name>.namespace" or something to set GIT_NAMESPACE
>> automatically when push to or fetch from remote server.
>>
>> Solution 2: passing the namespace in URL.
>>
>> Again use remote-ext style URL to access remote repository:
>>
>>     $ git remote add foo \
>>         'ext::ssh git@server git --namespace foo %s path/to/repo.git'
>>
>>     $ git remote add bar \
>>         'ext::ssh git@server git --namespace bar %s path/to/repo.git'
>>
>> But if the remote server use a limit shell (such as git-shell or
>> gitolite), the above URLs won't work. This is because these git
>> specific shell (git-shell or gitolite) do not like options.
>>
>> Solution 3: use custom receivepack and uploadpack. e.g.
>>
>>     [remote "foo"]
>>         url = ssh://git@server/path/to/repo.git
>>         receivepack = git --namespace foo receive-pack
>>         uploadpack = git --namespace foo upload-pack
>>         fetch = +refs/heads/*:refs/remotes/foo/*
>>
>>     [remote "bar"]
>>         url = ssh://git@server/path/to/repo.git
>>         receivepack = git --namespace bar receive-pack
>>         uploadpack = git --namespace bar upload-pack
>>         fetch = +refs/heads/*:refs/remotes/foo/*
>>
>> Just like solution 2, these settings won't work without a patched
>> git-shell or gitolite.
>
> Gitolite has a namespaces branch that handles namespaces as described in
> http://gitolite.com/gitolite/namespaces.html
>
> Briefly, it recognises that you can have a "main" repo off of which
> several developer might want to hang their logical repos.
>
> It also recognises that the actual names of the logical repos will
> follow some pattern that may include the name of the developer also, and
> provides a way to derive the name of the physical repo from the logical
> one.
>
> There is an example or two in that link.

I should add that the Gitolite model is: the user doesn't need to know
about namespaces, because namespaces are just things that the server
admin is setting up for his own reasons...

...typically because he anticipates several dozens of people cloning the
same repo into their namespace and so he expects to save a lot of disk
doing this).

So in this model we don't really need anything on the client side.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: Access different NAMESPACE of remote repo from client side

Jiang Xin
2013/11/15 Sitaram Chamarty <[hidden email]>:

> On 11/15/2013 07:55 PM, Sitaram Chamarty wrote:
>> On 11/15/2013 01:49 PM, Jiang Xin wrote:
>>> Just like solution 2, these settings won't work without a patched
>>> git-shell or gitolite.
>>
>> Gitolite has a namespaces branch that handles namespaces as described in
>> http://gitolite.com/gitolite/namespaces.html
>>
>> Briefly, it recognises that you can have a "main" repo off of which
>> several developer might want to hang their logical repos.
>>
>> It also recognises that the actual names of the logical repos will
>> follow some pattern that may include the name of the developer also, and
>> provides a way to derive the name of the physical repo from the logical
>> one.
>>
>> There is an example or two in that link.

I use gitolite everyday, and thank you for inventing it. I use gitolite v2 for
a long time, and migrated to v3 recently. Thank you for pointing out
there is a "namespace" feature branch, and I love the implementation
very much, it is so easy to map logical repos to one physical repo.

>
> I should add that the Gitolite model is: the user doesn't need to know
> about namespaces, because namespaces are just things that the server
> admin is setting up for his own reasons...
>

I want to say something that git-namespace is sometimes significant
for normal user not only for admin.

Some of my repos are managed by Topgit. With the help of topgit,
dependance of the topic branch (usually named as "t/feature-name")
are tracked by file ".topdeps" and a special ref named as
"refs/top-bases/t/feature-name". So there are many topic branches
and many useful special refs.

Every time when I want to update all topic branches to the new
upstream version, just before run "tg update", I want to take a
snapshoot of all branches and refs. Clone is heavy, so I want to use
light-weight namespace.

If the server is out of my control, it would be nice to let the user pass
namespace to the server from client side.

Indeed, I patched gitolite first as the follows, than I find out git-shell
needs to be patched too.


diff --git a/src/gitolite-shell b/src/gitolite-shell
index 7a27cc0..1c957da 100755
--- a/src/gitolite-shell
+++ b/src/gitolite-shell
@@ -101,7 +101,7 @@ sub main {
     my $user = $ENV{GL_USER} = shift @ARGV;

     # set up the repo and the attempted access
-    my ( $verb, $repo ) = parse_soc();    # returns only for git commands
+    my ( $verb, $option, $repo ) = parse_soc();    # returns only for
git commands
     sanity($repo);
     $ENV{GL_REPO} = $repo;
     my $aa = ( $verb =~ 'upload' ? 'R' : 'W' );
@@ -135,7 +135,11 @@ sub main {
         _system( "git", "http-backend" );
     } else {
         my $repodir = "'$rc{GL_REPO_BASE}/$repo.git'";
-        _system( "git", "shell", "-c", "$verb $repodir" );
+        my $cmd = $verb;
+        if ($option) {
+            $cmd =~ s/^(git)[- ](.*)$/$1 $option $2/;
+        }
+        _system( "git", "shell", "-c", "$cmd $repodir" );
     }
     trigger( 'POST_GIT', $repo, $user, $aa, 'any', $verb );
 }
@@ -146,13 +150,24 @@ sub parse_soc {
     my $soc = $ENV{SSH_ORIGINAL_COMMAND};
     $soc ||= 'info';

-    my $git_commands = "git-upload-pack|git-receive-pack|git-upload-archive";
-    if ( $soc =~ m(^($git_commands) '/?(.*?)(?:\.git(\d)?)?'$) ) {
-        my ( $verb, $repo, $trace_level ) = ( $1, $2, $3 );
+    my $git_commands = "upload-pack|receive-pack|upload-archive";
+    my ( $verb, $option, $repo, $trace_level );
+    if ( $soc =~
m(^git\s+(?:(--namespace)(?:=|\s+)(\S+)\s+)?(\S+)\s+'/?(.*?)(?:\.git(\d)?)?'$)
) {
+        $option = "$1=$2";
+        $verb = "git-$3";
+        $repo = $4;
+        $trace_level = $5;
+    } elsif ( $soc =~ m(^git[- ]($git_commands) '/?(.*?)(?:\.git(\d)?)?'$) ) {
+        $verb = "git-$1";
+        $repo = $2;
+        $trace_level = $3;
+    }
+
+    if ($verb) {
         $ENV{D} = $trace_level if $trace_level;
         _die "invalid repo name: '$repo'" if $repo !~ $REPONAME_PATT;
         trace( 2, "git command", $soc );
-        return ( $verb, $repo );
+        return ( $verb, $option, $repo );
     }

     # after this we should not return; caller expects us to handle it all here



--
Jiang Xin
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: Access different NAMESPACE of remote repo from client side

Sitaram Chamarty
On 11/16/2013 01:30 PM, Jiang Xin wrote:
> 2013/11/15 Sitaram Chamarty <[hidden email]>:
>> On 11/15/2013 07:55 PM, Sitaram Chamarty wrote:

[snip]

>> I should add that the Gitolite model is: the user doesn't need to know
>> about namespaces, because namespaces are just things that the server
>> admin is setting up for his own reasons...
>>
>
> I want to say something that git-namespace is sometimes significant
> for normal user not only for admin.

Sure.  I only meant in the model that I wrote that branch for.

But consider a slight change of syntax:

    repo dev/CREATOR/..*
        C   =   @team
        RW+ =   CREATOR
        R   =   @all
        option namespace.pattern-1 = dev/%/%/NS/* is @1/@3 in @2

Let's say linux and git are parent repos already created (maybe earlier
in the conf).

This conf will let you use URLs like

    dev/u1/git/NS/bar   (becomes namespace u1/bar in git)
    dev/u2/git/NS/baz   (becomes namespace u2/baz in git)

Yeah it looks like a kludge, but all I wanted to do was to show you that
it's not entirely true to say the client cannot control the namespace!
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html