Quantcast

Branch dependencies

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Branch dependencies

martin f krafft-2
Dear list,

We are trying to approach a functionality I call "branch
dependencies". Essentially, the idea is rooted in distro
development, but probably applies to normal development too.

For instance, you might have a feature branch off upstream. Once
upstream advances, you should merge upstream into your feature
branch (or rebase your feature branch) to ensure that you are
working on the right assumptions.

Another case might be a feature you want to write, which depends on
two (or more) feature branches of other people, e.g. say you need
a new configuration option introduced in feature branch
"conf-option", and you also base your work on the "speedup" branch,
because otherwise the software is too slow for your new feature. If
one or both feature branches advance, you should merge, as before.

It would be useful if Git could help you keep track of what needs to
be merged, especially as the number of feature branches and
dependencies increases. TopGit was Petr's answer to this challenge,
and it works fine, albeit it's a bit too complex and we find it
scaring new contributors, rather than making their lives easier.

Therefore I am investigating ways in which to simplify/improve
TopGit. In doing so, I discovered that you guys made a lot of
progress in Git since the last time I had time to really dive into
your tool.

If you would permit me, then please let me ask if you can think of
Git functionality that could be useful in achieving what we're
trying to do.

For instance, there is git-branch --set-upstream, which could be
useful, but it only seems to support one "dependency" (which is
usually the remote ref being tracked by a branch).

One challenge seems to be that a branching point has no information
about which of the children continues as mainline — this information
is only available in a project's workflow policy. For instance:

  o--o--o--●    upstream
      \
       o--●     feature

But his is actually just the same as

      ,o--●     feature
  o--o
      `o--●     upstream

and Git has no way to find out whether it is now "feature" that
needs a merge of "upstream" or vice versa.

It is thus necessary somehow to store the (project-specific)
dependency information, to be able to (automatically) determine that
"feature" needs an update in the above.

TopGit does this using a file in the worktree, but many of us find
this suboptimal.

I have had the following alternative ideas:

  1. a separate DAG, like Git notes. The problem is that this
     requires additional refspecs to be set up for merges and
     fetches;

  2. like (1.), but a ref in refs/heads/* (like pristine-tar). This
     could be considered ugly as it exposes too much implementation
     detail;

  3. information stored in the Git commit messages. Again, too much
     implementation detail exposed and ugly;

  4. additional Git commit headers — this is not supported at the
     moment (cf. commit generation discussion);

  5. orphan parent nodes to certain commits, in which these data can
     be stored, e.g.

       o--o--o--●
            /
           o

     To fetch these data, one would walk up the DAG until one finds
     a multi-parent commit with a parent having a specific format
     (somewhat brittle…)

     To me, this is the least offensive, but it does expose
     implementation details in the commit history.

How else could I store the dependency information, keeping in mind
that I might have more than one dependency?

And the original question remains: given such dependency
information, which Git tools could I harness for the purpose, trying
to reduce the amount of additional code needed?

Thanks,

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
it is better to have loft and lost
than to never have loft at all.
                                                       -- groucho marx
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Branch dependencies

Bert Wesarg
Hi,

On Mon, Aug 1, 2011 at 14:19, martin f krafft <[hidden email]> wrote:
> Dear list,

while I appreciate, that you dig this topic up. I think you are trying
to solve the wrong problem first. My main problem with the TopGit
approach is, that you can't freely change the dependencies of a topic.
This may be not the most common case in distro development. But in my
eyes more problematic than maintaining the meta data.

Please note, that I'm more than aware the the TopGit approach for
handling the meta data is awful and we need a new way here. My
personal impression is, that the git notes is the best we can have.

For my first mentioned problem, I think a new 'system' needs to be
'rebase' based, not merge based like TopGit.

Bert
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Branch dependencies

martin f krafft-2
also sprach Bert Wesarg <[hidden email]> [2011.08.02.1506 +0200]:
> while I appreciate, that you dig this topic up. I think you are trying
> to solve the wrong problem first. My main problem with the TopGit
> approach is, that you can't freely change the dependencies of a topic.
> This may be not the most common case in distro development. But in my
> eyes more problematic than maintaining the meta data.

Hello Bert, thank you for taking the time to respond!

Could you please try to illuminate me a bit on a use-case of
changing dependencies? I am aware that TopGit has had a problem with
changing dependencies due to renamed branches, and I think I have
a solution to that (encode the dependent ref, not the branch head),
but I cannot come up with a use case for freely changing
dependencies just like that.

> For my first mentioned problem, I think a new 'system' needs to be
> 'rebase' based, not merge based like TopGit.

The problem with rebasing is that you cannot publish the branches.

However, maybe I am simply not seeing the light here. Do you have
some further ideas about what this would be like? Please keep in
mind that what I seek is not just a way to bring feature branches
up-to-date with upstream, but also to have those branches be shared
among developers.

Thanks,

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
"gott ist tot! und wir haben ihn getötet."
                                                 - friedrich nietzsche
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Branch dependencies

Bert Wesarg
Hi,

On Tue, Aug 2, 2011 at 21:08, martin f krafft <[hidden email]> wrote:

> also sprach Bert Wesarg <[hidden email]> [2011.08.02.1506 +0200]:
>> while I appreciate, that you dig this topic up. I think you are trying
>> to solve the wrong problem first. My main problem with the TopGit
>> approach is, that you can't freely change the dependencies of a topic.
>> This may be not the most common case in distro development. But in my
>> eyes more problematic than maintaining the meta data.
>
> Hello Bert, thank you for taking the time to respond!
>
> Could you please try to illuminate me a bit on a use-case of
> changing dependencies? I am aware that TopGit has had a problem with
> changing dependencies due to renamed branches, and I think I have
> a solution to that (encode the dependent ref, not the branch head),
> but I cannot come up with a use case for freely changing
> dependencies just like that.

Not each feature branch may end up in master. And there does not need
to be one feature branch which depends on all other features. For the
latter you have probably an empty feature branch which just depends on
all features. I call this branch mostly 'tip'. Removing a feature
branch from the tips dependency list only with merges can't be done
right now, and the proposed solutions never reached a usable state.

My second usecase is to convert a big quilt patch series into TopGit.
Such big Quilt patches have mostly an artificial dependency to its
predecessors. Removing these artifical dependencies makes it necessary
to remove dependencies from patches.

>
>> For my first mentioned problem, I think a new 'system' needs to be
>> 'rebase' based, not merge based like TopGit.
>
> The problem with rebasing is that you cannot publish the branches.

That doesn't hold you back to publish them. But the other side need to
know how to deal with them.

Saying that doesn't mean I know a good way to deal with them. I mostly
end up using plumbing commands to deal with this.

>
> However, maybe I am simply not seeing the light here. Do you have
> some further ideas about what this would be like? Please keep in
> mind that what I seek is not just a way to bring feature branches
> up-to-date with upstream, but also to have those branches be shared
> among developers.

I think that having the TopGit philosophy of one feature branch is one
patch, you can handle an rebased upstream. Thinking of a feature
branch as a series of patches makes this way harder. But I would like
to have this philosophy.

Bert

>
> Thanks,
>
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

changing the set of dependencies (was: Branch dependencies)

martin f krafft-2
also sprach Bert Wesarg <[hidden email]> [2011.08.03.0125 +0200]:

> Not each feature branch may end up in master. And there does not need
> to be one feature branch which depends on all other features. For the
> latter you have probably an empty feature branch which just depends on
> all features. I call this branch mostly 'tip'. Removing a feature
> branch from the tips dependency list only with merges can't be done
> right now, and the proposed solutions never reached a usable state.
>
> My second usecase is to convert a big quilt patch series into TopGit.
> Such big Quilt patches have mostly an artificial dependency to its
> predecessors. Removing these artifical dependencies makes it necessary
> to remove dependencies from patches.
Dear Bert, thank you for your reply.

Okay, understood. Yes, I agree entirely, the set of dependencies
needs to be mutable. And they must not be invalidated by branch
renames. Therefore, we really need some sort of other way to
identify branches.

Is there a way other than lamenting that refs did not get UUIDs
assigned to them from the early days onwards? ;)

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
"if one cannot enjoy reading a book over and over again,
 there is no use in reading it at all."
                                                        -- oscar wilde
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

TopGit with rebased branches, problems with publishing (was: Branch dependencies)

martin f krafft-2
In reply to this post by Bert Wesarg
also sprach Bert Wesarg <[hidden email]> [2011.08.03.0125 +0200]:

> >> For my first mentioned problem, I think a new 'system' needs to be
> >> 'rebase' based, not merge based like TopGit.
> >
> > The problem with rebasing is that you cannot publish the branches.
>
> That doesn't hold you back to publish them. But the other side need to
> know how to deal with them.
>
> Saying that doesn't mean I know a good way to deal with them. I mostly
> end up using plumbing commands to deal with this.
Let me address this in terms of patch queue branches. Obviously, you
are talking topic branches, but wrt rebase-publish, the two are
identical. Patch queue branches are essentially quilt imports, e.g.
one-commit-one-patch, usually edited with git rebase -i (which some
people consider to be the better quilt).

The problem that led me away from patch queue branches is that
— albeit unlikely in our context (distro packaging) — it is possible
that you and I both rewrite the patch queue at the same time. The
first one to push will then have his changes overwritten by the
second push — in order to push the rebased ref,
receive.denyNonFastForwards needs to be off.

Unfortunately, I can't see a remedy to this, apart from tagging each
and every patch queue branch, but that does not solve the problem of
having to merge changes made simultaneously. It could go something
like this:

  1. ensure that noone has pushed an updated ref since you last
     fetched.

  2. if a new ref is found, download it and rebase your own patch
     queue on top of it, consolidating the changes.

  3. tag and push your new ref.

But there's a race condition in this, and so we're back to square
one.

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
uʍop ǝpısdn sı ɹoʇıuoɯ ɹnoʎ
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Tracking topic branches with rebases vs. merges (was: Branch dependencies)

martin f krafft-2
In reply to this post by Bert Wesarg
also sprach Bert Wesarg <[hidden email]> [2011.08.03.0125 +0200]:
> I think that having the TopGit philosophy of one feature branch is
> one patch, you can handle an rebased upstream. Thinking of
> a feature branch as a series of patches makes this way harder. But
> I would like to have this philosophy.

Let me just make sure I understand you right: you do not like the
following branch management strategy:

       o--o--o--+--o--o--+--o-.
      /        /        /      \
  o--o--o--o--o--o--o--o--o--o--o--●

and you would prefer if the topic branch was rebased all along (like
what git.git advocates), and then transferred to mainline with
git-format-patch/git-send-e-mail/git-am (or ff-merged).

I am torn on this issue. On the one hand, I do not find the above to
be so problematic, especially not if the downstream merges happen
only infrequently (undo merges that do not produce conflicts, as
advocated by gitworkflows(7)).

On the other hand, I completely agree with you that rebasing is much
nicer, and it certainly works without publishing the branch. In
git.git, people seem to maintain their own branches and send patch
sets to the mailing list for review.

But how could you and I truly cooperate on a feature branch without
using merges, and without establishing a lock-unlock protocol to
ensure only sequential updates of the refs?

Thanks,

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
"alles gackert, aber wer will noch still
 auf dem nest sitzen und eier zu brüten?"
                                      -- friedrich wilhelm nietzsche
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Branch dependencies

Nicolas Sebrecht-3
In reply to this post by martin f krafft-2
The 02/08/11, martin f krafft wrote:
> also sprach Bert Wesarg <[hidden email]> [2011.08.02.1506 +0200]:

> > For my first mentioned problem, I think a new 'system' needs to be
> > 'rebase' based, not merge based like TopGit.
>
> The problem with rebasing is that you cannot publish the branches.

But you may make public the way to set up the builded branch. Say you
share A and B branches and you build M branch on top of them. You are
able to build M as long as others know M rely on A and B.

In a situation like

      a--+       (A)
         M--y--z (M)
      b--+       (B)

or

   a--+          (A)
      b--+       (B)
         M--y--z (M)

M is the builded branch. Commits y and z should not be public, IMHO.

--
Nicolas Sebrecht
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Loading...