Quantcast

[RFD] Rewriting safety - warn before/when rewriting published history

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
Git includes protection against rewriting published history on the
receive side with fast-forward check by default (which can be
overridden) and various receive.deny* configuration variables,
including receive.denyNonFastForwards.

Nevertheless git users requested (among others in Git User's Survey)
more help on creation side, namely preventing rewriting parts of
history which was already made public (or at least warning that one is
about to rewrite published history).  The "warn before/when rewriting
published history" answer in "17. Which of the following features would
you like to see implemented in git?" multiple-choice question in latest
Git User's Survey 2011[1] got 24% (1525) responses.

[1]: https://www.survs.com/results/Q5CA9SKQ/P7DE07F0PL

So people would like for git to warn them about rewriting history before
they attempt a push and it turns out to not fast-forward.


What prompted this email is the fact that Mercurial includes support for
tracking which revisions (changesets) are safe to modify in its 2.1
latest version:

  http://lwn.net/Articles/478795/
  http://mercurial.selenic.com/wiki/WhatsNew

It does that by tracking so called "phase" of a changeset (revision).

  http://mercurial.selenic.com/wiki/Phases
  http://mercurial.selenic.com/wiki/PhasesDevel

  http://www.logilab.org/blogentry/88203
  http://www.logilab.org/blogentry/88219
  http://www.logilab.org/blogentry/88259
 

While we don't have to play catch-up with Mercurial features, I think
something similar to what Mercurial has to warn about rewriting
published history (amend, rebase, perhaps even filter-branch) would
be nice to have.  Perhaps even follow UI used by Mercurial, and/or
translating its implementation into git terms.

In Mercurial 2.1 there are three available phases: 'public' for
published commits, 'draft' for local un-published commits and
'secret' for local un-published commits which are not meant to
be published.

The phase of a changeset is always equal to or higher than the phase
of it's descendants, according to the following order:

      public < draft < secret

Commits start life as 'draft', and move to 'public' on push.

Mercurial documentation talks about phase of a commit, which might
be a good UI, ut also about commits in 'public' phase being "immutable".
As commits in Git are immutable, and rewriting history is in fact
re-doing commits, this description should probably be changed.

While default "push matching" behavior makes it possible to have
"secret" commits, being able to explicitly mark commits as not for
publishing might be a good idea also for Git.


What do you think about this?
--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Ben Walton
Excerpts from Jakub Narebski's message of Sat Feb 04 14:45:53 -0500 2012:

Hi Jakub,

These items are as much about UI as anything else, I think.  UI that
better helps users to know the state of their commits and branches can
only be a good thing.  People that have used git for a while and are
comfortable with it may not see the need/point of these, but I think
they could both really help new users.

> In Mercurial 2.1 there are three available phases: 'public' for
> published commits, 'draft' for local un-published commits and
> 'secret' for local un-published commits which are not meant to be
> published.

How do you envision such a feature in git?

A 'draft' commit (or chain of commits) could be determined from the
push matching definitions and then marked with simple decorations in
log output...This would extend the ability of status to note that your
are X commits ahead of foo.  This would see any commit on a branch
that would be pushed automatically decorated with a 'draft' status.

> While default "push matching" behavior makes it possible to have
> "secret" commits, being able to explicitly mark commits as not for
> publishing might be a good idea also for Git.

Do you see using configuration or convention to achieve this?

For example, any branch named private/foo could, by convention, be
un-pushable without a force option?  Alternately, a config item
similar to the push matching stuff to allow the users to designate
un-pushable branches could work too.

Please don't take the above implementation possibilities as anything
more than a starting point for discussion as they may be deeply
flawed.  I'm just tossing a few things out there as I think this is a
good discussion to have.

Thanks
-Ben
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
On Sun, 5 Feb 2012, Ben Walton wrote:
> Excerpts from Jakub Narebski's message of Sat Feb 04 14:45:53 -0500 2012:
>
> Hi Jakub,
>
> These items are as much about UI as anything else, I think.  UI that
> better helps users to know the state of their commits and branches can
> only be a good thing.  People that have used git for a while and are
> comfortable with it may not see the need/point of these, but I think
> they could both really help new users.

As I said, 1500+ git users would like to have such feature, according
to latest Git User's Survey.

> > In Mercurial 2.1 there are three available phases: 'public' for
> > published commits, 'draft' for local un-published commits and
> > 'secret' for local un-published commits which are not meant to be
> > published.
>
> How do you envision such a feature in git?
>
> A 'draft' commit (or chain of commits) could be determined from the
> push matching definitions and then marked with simple decorations in
> log output...This would extend the ability of status to note that your
> are X commits ahead of foo.  This would see any commit on a branch
> that would be pushed automatically decorated with a 'draft' status.

I think that in its basic form (treating all remotes equally) commits
in 'public' phase would be those reachable from remote-tracking branches.
Otherwise commits would be in 'draft' phase, unless explicitly marked
as 'secret' (it we implement 'secret' phase, that is).

The safety new I think of would (similarly to Mercurial phases) prevent
or warn about amending published commit, and rebasing commits which were
already published (in 'public' phase).  That would require modifications
to git-commit and git-amend, I think...

Maybe even Git could refuse or warn on the local side about non
fast-forward update of public branch, to help users of third-party tools.
 

> > While default "push matching" behavior makes it possible to have
> > "secret" commits, being able to explicitly mark commits as not for
> > publishing might be a good idea also for Git.
>
> Do you see using configuration or convention to achieve this?
>
> For example, any branch named private/foo could, by convention, be
> un-pushable without a force option?  Alternately, a config item
> similar to the push matching stuff to allow the users to designate
> un-pushable branches could work too.

I'm not sure, but the config item might be a good solution.  Git would
skip publishing 'secret' commits (commits from 'secret' branch) if it
would otherwise publish it due to glob refspec, and refuse (or warn)
publishing 'secret' branches explicitly.

Currently if you use default "push matching", then those branches that
you didn't push explicitly wouldn't be pushed.  But that does not prevent
pushing them by accident, and does not give UI to check if branch is
private or not (e.g. to use in git-aware shell prompt).

--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
In reply to this post by Jakub Narębski
Please don't remove git mailing list from Cc... Oh, I see that you
forgot to send to list, but resend your email there.

On Sun, 5 Feb 2012, Philip Oakley wrote:
> From: "Jakub Narebski" <[hidden email]>
> Sent: Saturday, February 04, 2012 7:45 PM

> > Git includes protection against rewriting published history on the
> > receive side with fast-forward check by default (which can be
> > overridden) and various receive.deny* configuration variables,
> > including receive.denyNonFastForwards.
> >
> > Nevertheless git users requested (among others in Git User's Survey)
> > more help on creation side, namely preventing rewriting parts of
> > history which was already made public (or at least warning that one is
> > about to rewrite published history).  The "warn before/when rewriting
> > published history" answer in "17. Which of the following features would
> > you like to see implemented in git?" multiple-choice question in latest
> > Git User's Survey 2011[1] got 24% (1525) responses.
> >
> > [1]: https://www.survs.com/results/Q5CA9SKQ/P7DE07F0PL
> >
> > So people would like for git to warn them about rewriting history before
> > they attempt a push and it turns out to not fast-forward.
>
> Another area that is implicitly related is that of (lack of) publication of
> sub-module updates. A mechanisms that, in the super project, knows the
> status of the (local) submodules, such as where they would be sourced from,
> i.e. what was last pushed & where, could help in such instances.

"Better support for submodules" had almost the same number of requests
in the latest Git User's Survey 2011 (25% which means 1582 responses).
 
Remembering when to do recursive push and where would be a very nice thing.

[...]
> Recording where they were pushed to would be useful for synchronising
> sub-modules and their super projects. That is, giving remote users a clue as
> to where they might find mising sub-modules.

Is it a matter of correctly writing configuration with current git?
I don't use submodules myself, so I cannot say.

> > Mercurial documentation talks about phase of a commit, which might
> > be a good UI, ut also about commits in 'public' phase being "immutable".
> > As commits in Git are immutable, and rewriting history is in fact
> > re-doing commits, this description should probably be changed.
> >
> > While default "push matching" behavior makes it possible to have
> > "secret" commits, being able to explicitly mark commits as not for
> > publishing might be a good idea also for Git.
> >
>
> Being able to mark temporary, out of sequence or other hacks as Secret could
> be useful, as would recording where Public commits had been sent.

Marking as 'secret' must I think be explicit, but I think 'public' phase
should be inferred from remote-tracking branches.  The idea of phases is
to allow UI to ask about status of commits: can we amend / rebase it or
not, can we push it or not.

--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Johan Herland
2012/2/5 Jakub Narebski <[hidden email]>:
>> Being able to mark temporary, out of sequence or other hacks as Secret could
>> be useful, as would recording where Public commits had been sent.
>
> Marking as 'secret' must I think be explicit, but I think 'public' phase
> should be inferred from remote-tracking branches.  The idea of phases is
> to allow UI to ask about status of commits: can we amend / rebase it or
> not, can we push it or not.

I agree that the 'public' state should (by default) be automatically
inferred from remote-tracking branches. As it stands, we can do this
with current git, by writing a pre-rebase hook that checks if any of
the commits to-be-rebased are reachable from any remote-tracking
branch.

Unfortunately, the pre-rebase hook only affects 'git rebase', and in
order to do the same check on 'git commit --amend' you'd have to write
a similar pre-commit hook (don't know how easy it is to find the
amended commit from within the hook). Maybe we should add a
pre-rewrite hook that trigger in the same situations as the
post-rewrite hook.

This should take care of the simplest 'public' use case in a
push-based workflow. If you publish commits by other means
(send-email, bundles, pulling directly from your repo), you need some
other way to mark the 'public' commits. One solution would be using
'git notes' to annotate the 'public' commits on a 'refs/notes/public'
notes ref. Your pre-rebase/pre-rewrite hook should then check if any
of the commits to-be-rewritten are reachable from any commit annotated
as 'public'.

Also, if you want to record where 'public' commits have been sent
(other than what can be inferred from the remote-tracking branches),
you could write this into the refs/notes/public annotation.

As for 'secret' commits, you could annotate these on a
refs/notes/secret notes ref, and then teach 'git push' (or whatever
other method for publishing commits you use) to refuse to publish
commits annotated on this notes ref. Possibly we would want to add a
"pre-push" or "pre-publish" hook.


Have fun! :)

...Johan

--
Johan Herland, <[hidden email]>
www.herland.net
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
On Sun, 5 Feb 2012, Johan Herland wrote:
> 2012/2/5 Jakub Narebski <[hidden email]>:

> > > Being able to mark temporary, out of sequence or other hacks as Secret could
> > > be useful, as would recording where Public commits had been sent.
> >
> > Marking as 'secret' must I think be explicit, but I think 'public' phase
> > should be inferred from remote-tracking branches.  The idea of phases is
> > to allow UI to ask about status of commits: can we amend / rebase it or
> > not, can we push it or not.
>
> I agree that the 'public' state should (by default) be automatically
> inferred from remote-tracking branches. As it stands, we can do this
> with current git, by writing a pre-rebase hook that checks if any of
> the commits to-be-rebased are reachable from any remote-tracking
> branch.

It is nice that we can achieve a large part of this feature with existing
infrastructure.  It would be nice if we ship such pre-rebase hook with
git, so people can just enable it if they want to use this functionality,
like the default pre-commit hook that checks for whitespace errors.
 
> Unfortunately, the pre-rebase hook only affects 'git rebase', and in
> order to do the same check on 'git commit --amend' you'd have to write
> a similar pre-commit hook (don't know how easy it is to find the
> amended commit from within the hook). Maybe we should add a
> pre-rewrite hook that trigger in the same situations as the
> post-rewrite hook.

pre-rewrite hook would be a really nice to have, especially that it would
(I hope) cover third party tools like various GUIs for git; and also
git-filter-branch.

Note however that the safety net, i.e. refusing or warning against attempted
rewrite of published history is only part of issue.  Another important part
is querying and showing "phase" of a commit.  What I'd like to see is
ability to show among others in "git log" and "git show" output if commit
was already published or not (and if it is marked 'secret').

> This should take care of the simplest 'public' use case in a
> push-based workflow. If you publish commits by other means
> (send-email, bundles, pulling directly from your repo), you need some
> other way to mark the 'public' commits. One solution would be using
> 'git notes' to annotate the 'public' commits on a 'refs/notes/public'
> notes ref. Your pre-rebase/pre-rewrite hook should then check if any
> of the commits to-be-rewritten are reachable from any commit annotated
> as 'public'.

Another solution would be to create "fake" remote-tracking branches
by git-bundle and git-send-email.
 
> Also, if you want to record where 'public' commits have been sent
> (other than what can be inferred from the remote-tracking branches),
> you could write this into the refs/notes/public annotation.

I wonder if this too can be done by hook...
 
> As for 'secret' commits, you could annotate these on a
> refs/notes/secret notes ref, and then teach 'git push' (or whatever
> other method for publishing commits you use) to refuse to publish
> commits annotated on this notes ref. Possibly we would want to add a
> "pre-push" or "pre-publish" hook.

Well, addition of pre-push / pre-publish was resisted on the grounds
that all it does is something that can be as easy done by hand before
push.  Perhaps this new use case would help bring it forward, don't
you think?

Thanks for all the comments.
--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Johan Herland
On Sun, Feb 5, 2012 at 21:46, Jakub Narebski <[hidden email]> wrote:

> On Sun, 5 Feb 2012, Johan Herland wrote:
>> 2012/2/5 Jakub Narebski <[hidden email]>:
>> > > Being able to mark temporary, out of sequence or other hacks as Secret could
>> > > be useful, as would recording where Public commits had been sent.
>> >
>> > Marking as 'secret' must I think be explicit, but I think 'public' phase
>> > should be inferred from remote-tracking branches.  The idea of phases is
>> > to allow UI to ask about status of commits: can we amend / rebase it or
>> > not, can we push it or not.
>>
>> I agree that the 'public' state should (by default) be automatically
>> inferred from remote-tracking branches. As it stands, we can do this
>> with current git, by writing a pre-rebase hook that checks if any of
>> the commits to-be-rebased are reachable from any remote-tracking
>> branch.
>
> It is nice that we can achieve a large part of this feature with existing
> infrastructure.  It would be nice if we ship such pre-rebase hook with
> git, so people can just enable it if they want to use this functionality,
> like the default pre-commit hook that checks for whitespace errors.

Yeah. As it is, the pre-rebase hook shipped with v1.7.9 (when
activated) does something similar (i.e. prevent rewriting 'public'
commits). However, it's highly workflow-specific, since it determines
whether the branch being rebased has been merged into "next" or
"master". IMHO, a hook that tested for reachability from
remote-tracking refs would be more generally useful. Obviously, the
two can be combined, and even further combinations may be desirable
(e.g. also checking for reachability from commits annotated in
refs/notes/public).

>> Unfortunately, the pre-rebase hook only affects 'git rebase', and in
>> order to do the same check on 'git commit --amend' you'd have to write
>> a similar pre-commit hook (don't know how easy it is to find the
>> amended commit from within the hook). Maybe we should add a
>> pre-rewrite hook that trigger in the same situations as the
>> post-rewrite hook.
>
> pre-rewrite hook would be a really nice to have, especially that it would
> (I hope) cover third party tools like various GUIs for git; and also
> git-filter-branch.
>
> Note however that the safety net, i.e. refusing or warning against attempted
> rewrite of published history is only part of issue.  Another important part
> is querying and showing "phase" of a commit.  What I'd like to see is
> ability to show among others in "git log" and "git show" output if commit
> was already published or not (and if it is marked 'secret').

Today, you can use --decorate to display remote-tracking refs in the
log/show output. However, only the tip commits are decorated, so if
the commits shown are not at the tip, you're out of luck. I believe
teaching log/show to decorate _all_ commits that are reachable from
some given ref(s) should be fairly straightforward.

If you use 'git notes' to annotate 'public' and 'secret' states, then
you can also use the --show-notes=<ref> option to let show/log display
the annotations on 'public'/'secret' commits.

>> This should take care of the simplest 'public' use case in a
>> push-based workflow. If you publish commits by other means
>> (send-email, bundles, pulling directly from your repo), you need some
>> other way to mark the 'public' commits. One solution would be using
>> 'git notes' to annotate the 'public' commits on a 'refs/notes/public'
>> notes ref. Your pre-rebase/pre-rewrite hook should then check if any
>> of the commits to-be-rewritten are reachable from any commit annotated
>> as 'public'.
>
> Another solution would be to create "fake" remote-tracking branches
> by git-bundle and git-send-email.

Good point. Creating such "fake" remote-tracking branches might be a
good idea in those workflows anyway, simply to keep track of what has
been shared, and where.

>> Also, if you want to record where 'public' commits have been sent
>> (other than what can be inferred from the remote-tracking branches),
>> you could write this into the refs/notes/public annotation.
>
> I wonder if this too can be done by hook...

You're looking for someting like a post-push hook that runs on the
_client_ after a successful push. AFAIK, that doesn't exist yet. (Not
to be confused with the receive/update hooks that run on the
_server_.)

>> As for 'secret' commits, you could annotate these on a
>> refs/notes/secret notes ref, and then teach 'git push' (or whatever
>> other method for publishing commits you use) to refuse to publish
>> commits annotated on this notes ref. Possibly we would want to add a
>> "pre-push" or "pre-publish" hook.
>
> Well, addition of pre-push / pre-publish was resisted on the grounds
> that all it does is something that can be as easy done by hand before
> push.  Perhaps this new use case would help bring it forward, don't
> you think?

Maybe. I didn't follow the original discussion. From my POV, you could
argue that instead of another hook, you could always write a script
that does the 'secret' check before invoking 'git push', and then
you'd use that script instead of 'git push'. But you could argue the
same point for pretty much all of the other existing hooks (e.g.
instead of a pre-commit hook you could have your own commit wrapper
script). So I don't think that's a sufficient argument to refuse the
existence of a pre-push/publish hook.


Have fun! :)

...Johan

--
Johan Herland, <[hidden email]>
www.herland.net
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Steven Michalske
In reply to this post by Jakub Narębski
See inlined responses below.

On Feb 4, 2012, at 11:45 AM, Jakub Narebski wrote:

> So people would like for git to warn them about rewriting history before
> they attempt a push and it turns out to not fast-forward.
>

I like this idea and I encounter this issue with my co-workers new to git.
It scares them thinking they broke the repository.

> In Mercurial 2.1 there are three available phases: 'public' for
> published commits, 'draft' for local un-published commits and
> 'secret' for local un-published commits which are not meant to
> be published.
>
> The phase of a changeset is always equal to or higher than the phase
> of it's descendants, according to the following order:
>
>      public < draft < secret

Let's not limit ourselves to just three levels.  They are a great start but I propose the following.

published - The commits that are on a public repository that if are rewritten will invoke uprisings.
        general rule here would be to revert or patch, no rewrites.
based - The commits that the core developers have work based upon. (not just the commits in their repo.)
        general rule is notify your fellow developers before a rewrite.
shared - The commits that are known to your fellow core developers.
        These commits are known, but have not had work based off of them.  Minimal risk to rewrite.
local - The commits that are local only, no one else has a copy.
        Commits your willing to share, but have not been yet shared, either from actions of you, or a fetch from others.
restricted or private - The commits that you do not want shared.
        Manually added, think of a branch tip marked as restricted automatically promotes commits to the branch as restricted.

Maybe make these like nice levels, but as two components, publicity 0-100 and rewritability 0-100
        Published is publicity 100 and rewritability 0
        Restricted is publicity 0 and rewritability 100
        Based publicity 75 and rewritability 25
        Shared publicity 50 and rewritability 50
        Local publicity 25 and rewritability 75
        Restricted publicity 0 and rewritability 100

Other option are flags stating if the commit is published, based, shared, or restricted.
You could have a published and based commit that is more opposed to rewrite than a public commit.

Call security on a published restricted commit ;-)

Commits are by default local.

Commits are published when they are pushed or fetched and merged to a publishing branch of a repository.
        On fetch/merge a post merge hook should send back a note to the remote repository that the commits were published.

Restricted commits/branches/tags should not be made public, error out and require clearing of the attribute or a --force-restricted option that automatically removes the restricted attribute.  They are at least promoted to shared, if not published.

Based is only used in situations where you have developers sharing amongst their repositories, and you want a rule that is less restrictive than no rewrites.

Shared is what we have now when a commit is in a remote repository without the no rewrite options. e.g. receive.denyNonFastForwards.

As it stands now we can infer local and shared,  we need metadata to know when a commit is made based, published, or restricted.


Using the nomenclature from Mercurial
>      public < draft < secret

public -> publicity 100, rewritability 0
draft -> publicity ?, rewritability 50
secret -> publicity 0, rewritability 100

Steve--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Johan Herland
On Mon, Feb 6, 2012 at 01:57, Steven Michalske <[hidden email]> wrote:

> On Feb 4, 2012, at 11:45 AM, Jakub Narebski wrote:
>> In Mercurial 2.1 there are three available phases: 'public' for
>> published commits, 'draft' for local un-published commits and
>> 'secret' for local un-published commits which are not meant to
>> be published.
>>
>> The phase of a changeset is always equal to or higher than the phase
>> of it's descendants, according to the following order:
>>
>>      public < draft < secret
>
> Let's not limit ourselves to just three levels.  They are a great start but I propose the following.
>
> published - The commits that are on a public repository that if are rewritten will invoke uprisings.
>        general rule here would be to revert or patch, no rewrites.
> based - The commits that the core developers have work based upon. (not just the commits in their repo.)
>        general rule is notify your fellow developers before a rewrite.
> shared - The commits that are known to your fellow core developers.
>        These commits are known, but have not had work based off of them.  Minimal risk to rewrite.
> local - The commits that are local only, no one else has a copy.
>        Commits your willing to share, but have not been yet shared, either from actions of you, or a fetch from others.
> restricted or private - The commits that you do not want shared.
>        Manually added, think of a branch tip marked as restricted automatically promotes commits to the branch as restricted.
>
> Maybe make these like nice levels, but as two components, publicity 0-100 and rewritability 0-100
>        Published is publicity 100 and rewritability 0
>        Restricted is publicity 0 and rewritability 100
>        Based publicity 75 and rewritability 25
>        Shared publicity 50 and rewritability 50
>        Local publicity 25 and rewritability 75
>        Restricted publicity 0 and rewritability 100
>
> [...]

With all due respect, I believe this is crazy. You're adding an entire
layer of complexity on top of commits that every user has to know
about, and has little or no value to most of them. IMHO, most users
only want Git to help them avoid doing something stupid (rewriting
'public' commits or publishing 'secret' commits), and to do so with
the minimal amount of manual user interaction. The above idea is more
suitable for armchair dictators that want to micromanage their commits
along two arbitrary axes of evil^H^H^H^Hpointlessness. On both axes,
you'll need threshold values where Git starts refusing to
publish/rewrite your commit. Hence, the only thing that matters is
whether the 'publicity'/'rewritability' value is above/below that
threshold, at which point you could save yourself a lot of trouble by
making them simple boolean flags instead.

Having said that, you can use 'git notes' along with existing and
proposed hooks (as described elsewhere in this thread) to implement
whatever crazy commit publishing/rewriting scheme you desire. To
misquote someone famous: I disapprove of what you want to do with Git,
but I will defend to the death your right to make Git do what you want
(in the privacy of your own repos). ;)


Have fun! :)

...Johan

--
Johan Herland, <[hidden email]>
www.herland.net
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
In reply to this post by Steven Michalske
On Mon, 6 Feb 2012, Steven Michalske wrote:

> See inlined responses below.

Is this comment necessary at all?
 
> On Feb 4, 2012, at 11:45 AM, Jakub Narebski wrote:
>
> > So people would like for git to warn them about rewriting history before
> > they attempt a push and it turns out to not fast-forward.
> >
>
> I like this idea and I encounter this issue with my co-workers new to git.
> It scares them thinking they broke the repository.

It is true that while this feature would be useful also for "power
users", it would be most helpful for newbies (users new to git).

So I am afraid that implementing it with example hooks that must be
turned on explicitly might be not enough...
 

> > In Mercurial 2.1 there are three available phases: 'public' for
> > published commits, 'draft' for local un-published commits and
> > 'secret' for local un-published commits which are not meant to
> > be published.
> >
> > The phase of a changeset is always equal to or higher than the phase
> > of it's descendants, according to the following order:
> >
> >      public < draft < secret
>
> Let's not limit ourselves to just three levels.  They are a great start
> but I propose the following.

As we don't have any implementation, I'd rather we don't multiply entities.
I was even thinking about limiting to just 'public' and 'draft' "phases".
 
> published - The commits that are on a public repository that if are
>       rewritten will invoke uprisings. general rule here would be
>       to revert or patch, no rewrites.
> based - The commits that the core developers have work based upon.
>       (not just the commits in their repo.)
> general rule is notify your fellow developers before a rewrite.
> shared - The commits that are known to your fellow core developers.
> These commits are known, but have not had work based off of them.
> Minimal risk to rewrite.

All these are very fairly nuanced, with minuscule differences between
them.  I'd rather not multiply entities, especially not introduce such
hard to guess what it about from their name.

In Mercurial phases share hierarchy of traits:
http://mercurial.selenic.com/wiki/Phases
 
           |        traits       |
           .......................  
           | immutable | shared  |
 ----------+-----------+---------+
 public    | x         | x       | ^
 draft     |           | x       | ^
 secret    |           |         | ^
       
The names of those traits probably should be changed in Git.

Those traits are boolean in Mercurial, but I think we can implement
what you would like to have to change them to tristate: 'deny' (unless
forced, i.e. the same as true), 'warn', 'ignore' (i.e. the same as false).

I think that it would be nice to be able to tune "severity" of trait
on per-remote and/or per-branch basis.  This way you would get warned
before rewriting commits that were pushed to your group repository,
and prevented from rewriting commits that are present in projects public
repository.

Nevertheless I think it is something better left for later, and added
only if it turns out to be really needed.

> local - The commits that are local only, no one else has a copy.
> Commits your willing to share, but have not been yet shared,
> either from actions of you, or a fetch from others.

That's Mercurial's 'draft' phase.

> restricted or private - The commits that you do not want shared.
> Manually added, think of a branch tip marked as restricted
> automatically promotes commits to the branch as restricted.

That's Mercurial 'secret' phase.

 
> Maybe make these like nice levels, but as two components,
> publicity 0-100 and rewritability 0-100
> Published is publicity 100 and rewritability 0
> Restricted is publicity 0 and rewritability 100
> Based publicity 75 and rewritability 25
> Shared publicity 50 and rewritability 50
> Local publicity 25 and rewritability 75
> Restricted publicity 0 and rewritability 100

Continuous traits are IMHO a bad idea.  You would have to quantize them
and turn them on into specific behavior: ignore, warn, deny.

For example WTF does 25 "publicity" (bad name) or "rewritability" actually
means in term of git behavior, eh?
 
> Other option are flags stating if the commit is published, based,
> shared, or restricted.  You could have a published and based commit
> that is more opposed to rewrite than a public commit.
>
> Call security on a published restricted commit ;-)

Please note that while "phases" look like they are trait of individual
commits, they are in fact artifact of revision walking.  The idea is
that ancestors of 'private' commit can be 'private', 'draft' or 'public',
that ancestors of 'draft' commit are 'draft' or 'public', and that _all_
ancestors of 'public' commit are 'public'.
 
> Commits are by default local.

This 'by default' needs to be specified further, because for example
all commits in freshly cloned repository should be in 'public' phase
by default.  

Also, don't say 'commits are local', 'commits are published'; use "phases"
nomenclature (at least until we invent something so much better that it
is worth breaking consistency with Mercurial terminology).
>
> Commits are published when they are pushed or fetched and merged to
> a publishing branch of a repository.

BTW. I am not sure if pushing to remote repository updates (or can update)
any remote-tracking branches...

> On fetch/merge a post merge hook should send back a note to
> the remote repository that the commits were published.

I think this is unnecessary in the "best practices" scenario, where each
user has separate private repository where he/she does his/her work, and
one's own public repository, where people fetch from.  He/she can push
to some shared repository, and that has to be supported too.

Though there is mothership/ sattellite situation, where you can pull and
push only from one direction.  There we might want for some way to notify
that some commits were fetched and should now be considered 'public'.
Though I am not sure if it is really necessary.
 
> Restricted commits/branches/tags should not be made public, error out and
> require clearing of the attribute or a --force-restricted option that
> automatically removes the restricted attribute.  They are at least promoted
> to shared, if not published.  

Or just skip them (silently or not) if we push using globbing refspec, and
glob matches some refs marked as 'private'.
 
> Based is only used in situations where you have developers sharing amongst
> their repositories, and you want a rule that is less restrictive than
> no rewrites.  

Multiplying entities.
 
> Shared is what we have now when a commit is in a remote repository without
> the no rewrite options. e.g. receive.denyNonFastForwards.

Multiplying entities.
 
[...]
> > Using the nomenclature from Mercurial
> >      public < draft < secret
>
> public -> publicity 100, rewritability 0
> draft -> publicity ?, rewritability 50
> secret -> publicity 0, rewritability 100

That doesn't really help, at all.

--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
In reply to this post by Johan Herland
On Sun, 5 Feb 2012, Johan Herland wrote:
> On Sun, Feb 5, 2012 at 21:46, Jakub Narebski <[hidden email]> wrote:
>> On Sun, 5 Feb 2012, Johan Herland wrote:
>>> 2012/2/5 Jakub Narebski <[hidden email]>:

[...]

>>> I agree that the 'public' state should (by default) be automatically
>>> inferred from remote-tracking branches. As it stands, we can do this
>>> with current git, by writing a pre-rebase hook that checks if any of
>>> the commits to-be-rebased are reachable from any remote-tracking
>>> branch.
>>
>> It is nice that we can achieve a large part of this feature with existing
>> infrastructure.  It would be nice if we ship such pre-rebase hook with
>> git, so people can just enable it if they want to use this functionality,
>> like the default pre-commit hook that checks for whitespace errors.
>
> Yeah. As it is, the pre-rebase hook shipped with v1.7.9 (when
> activated) does something similar (i.e. prevent rewriting 'public'
> commits). However, it's highly workflow-specific, since it determines
> whether the branch being rebased has been merged into "next" or
> "master". IMHO, a hook that tested for reachability from
> remote-tracking refs would be more generally useful. Obviously, the
> two can be combined, and even further combinations may be desirable
> (e.g. also checking for reachability from commits annotated in
> refs/notes/public).

Relying on (default) hooks to implement this feature has the disadvantage
that it wouldn't be turned on by default... while this feature would be
most helpful for users new to git (scared by refuse to push).
 
I am not sure either if everything (wrt. safety net) can be implemented
via hooks.  One thing that I forgot about is preventing rewinding of
branch past the published commit using e.g. "git reset --hard <commit>".
Unless `pre-rewrite` hook could be used for that safety too...

[...]

>> Note however that the safety net, i.e. refusing or warning against attempted
>> rewrite of published history is only part of issue.  Another important part
>> is querying and showing "phase" of a commit.  What I'd like to see is
>> ability to show among others in "git log" and "git show" output if commit
>> was already published or not (and if it is marked 'secret').
>
> Today, you can use --decorate to display remote-tracking refs in the
> log/show output. However, only the tip commits are decorated, so if
> the commits shown are not at the tip, you're out of luck. I believe
> teaching log/show to decorate _all_ commits that are reachable from
> some given ref(s) should be fairly straightforward.

That would be nice.
 
> If you use 'git notes' to annotate 'public' and 'secret' states, then
> you can also use the --show-notes=<ref> option to let show/log display
> the annotations on 'public'/'secret' commits.

First, in my opinion annotating _all_ commits with their phase is I think
out of question, especially annotating 'public' commits.  I don't think
git-notes mechanism would scale well to annotating every commit; but
perhaps this was tested to work, and I am mistaken.
 
Second, I have doubts if "phase" is really state of an individual commit,
and not the feature of revision walking.

Take for example the situation where given commit is reference by
remote-tracking branch 'public/foo', and also by two local branches:
'foo' with upstream 'public/foo', and local branch 'bar' with no upstream.

Now it is quite obvious that this feature should prevent rewriting 'foo'
branch, for which commits are published upstream.  But what about branch
'bar'?  Should we prevent rewriting (e.g. rebase) here too?  What about
rewinding 'bar' to point somewhere else.  What if 'bar' is really detached
HEAD?

These questions need to be answered...

[...]

>>> Also, if you want to record where 'public' commits have been sent
>>> (other than what can be inferred from the remote-tracking branches),
>>> you could write this into the refs/notes/public annotation.
>>
>> I wonder if this too can be done by hook...
>
> You're looking for someting like a post-push hook that runs on the
> _client_ after a successful push. AFAIK, that doesn't exist yet. (Not
> to be confused with the receive/update hooks that run on the
> _server_.)

And such hook could react to what was successfully pushed.  Without
such hook we would have to write wrapper around git-push and parse
its output...
 
Nb. such hook could create "fake" remote-tracking branches if given
push-only remote doesn't have them set up.

>>> As for 'secret' commits, you could annotate these on a
>>> refs/notes/secret notes ref, and then teach 'git push' (or whatever
>>> other method for publishing commits you use) to refuse to publish
>>> commits annotated on this notes ref. Possibly we would want to add a
>>> "pre-push" or "pre-publish" hook.
>>
>> Well, addition of pre-push / pre-publish was resisted on the grounds
>> that all it does is something that can be as easy done by hand before
>> push.  Perhaps this new use case would help bring it forward, don't
>> you think?
>
> Maybe. I didn't follow the original discussion. From my POV, you could
> argue that instead of another hook, you could always write a script
> that does the 'secret' check before invoking 'git push', and then
> you'd use that script instead of 'git push'. But you could argue the
> same point for pretty much all of the other existing hooks (e.g.
> instead of a pre-commit hook you could have your own commit wrapper
> script). So I don't think that's a sufficient argument to refuse the
> existence of a pre-push/publish hook.

Right, This would be for this feature very much like pre-commit hook.

--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Johan Herland
On Mon, Feb 6, 2012 at 15:44, Jakub Narebski <[hidden email]> wrote:
> On Sun, 5 Feb 2012, Johan Herland wrote:
> Relying on (default) hooks to implement this feature has the disadvantage
> that it wouldn't be turned on by default... while this feature would be
> most helpful for users new to git (scared by refuse to push).

True. I too believe that this will be most helpful if it is enabled by
default. That said, the easiest way to get there might be through
first demonstrating that it works in practice when implemented as
hooks.

> I am not sure either if everything (wrt. safety net) can be implemented
> via hooks.  One thing that I forgot about is preventing rewinding of
> branch past the published commit using e.g. "git reset --hard <commit>".
> Unless `pre-rewrite` hook could be used for that safety too...

Hmm. I don't think we'll be able to "plug" all the holes that might
leave the user in a rewritten state (e.g. what if the user (possibly
with the help of some tool) does an "echo $SHA1 >
.git/refs/head/master"?). And trying to plug too many holes might end
up annoying more experienced users who "know what they're doing".
Instead we might want to add a client-side check at push time. I
realize that this check is already done by the remote end, but the
client-side might be able to give a more helpful response along the
lines of:

  You are trying to push branch X to remote Y, but remote Y already
has a branch X that is N commits in front of you. You may want to
rebase your work on top of the remote branch (see 'git pull
--rebase'), If you instead force this push (with --force), you will
remove those N commits, and replace them with the M last commits from
your branch X.

  (followed by a list of the remote N and local M commits, respectively)

[...]

>> If you use 'git notes' to annotate 'public' and 'secret' states, then
>> you can also use the --show-notes=<ref> option to let show/log display
>> the annotations on 'public'/'secret' commits.
>
> First, in my opinion annotating _all_ commits with their phase is I think
> out of question, especially annotating 'public' commits.  I don't think
> git-notes mechanism would scale well to annotating every commit; but
> perhaps this was tested to work, and I am mistaken.

First, we don't need to annotate _all_ commits. For the 'public'
state, we only annotate the last/tip commit that was pushed/published.
From there, we can defer that all ancestor commits are also 'public'.
For the 'secret' state, we do indeed annotate _all_ secret commits,
but I believe this will be a somewhat limited number of commits. If
your workflow forces you to annotate millions of commits as 'secret',
I claim there is something wrong with your workflow.

Second, git-notes were indeed designed scale well to handle a large
number of notes, up to the same order of magnitude as the number of
commits in your repo. (When git-notes was originally written, I
successfully tested it on versions of a linux-kernel repo where every
single commit was annotated). In this case, the number of 'public'
annotations in your repo would be equal to the number of pushes you
do, and the number of 'secret' annotations would be equal to the
number of 'secret' commits in your repo. I'd expect both of these
numbers to be orders of magnitude smaller than the total number of
commits in your repo (given a fairly typical workflow in a fairly
typical repo).

> Second, I have doubts if "phase" is really state of an individual commit,
> and not the feature of revision walking.

I believe the 'public' state is a "feature of revision walking" (i.e.
one annotated 'public' commit implies that all its ancestors are also
'public'). However, the 'secret' state should be bound to the
individual commit, IMHO.

> Take for example the situation where given commit is reference by
> remote-tracking branch 'public/foo', and also by two local branches:
> 'foo' with upstream 'public/foo', and local branch 'bar' with no upstream.
>
> Now it is quite obvious that this feature should prevent rewriting 'foo'
> branch, for which commits are published upstream.  But what about branch
> 'bar'?  Should we prevent rewriting (e.g. rebase) here too?  What about
> rewinding 'bar' to point somewhere else.  What if 'bar' is really detached
> HEAD?
>
> These questions need to be answered...

Good point. There are two questions we may need to answer: "Has commit
X ever been published?", and "Has commit X ever been published in the
context of branch Y?". In the latter case, we do indeed need to take
the upstream branch into account.

Basically, there are three different "levels" for this rewrite/publish
protection to run at:

1. Do not meddle at all. This is the current behavior, and assumes
that if the user rewrites and pushes something, the user knows what
he/she is doing, and Git should not meddle (obviously unless the
server refuses the push).

2. Warn/refuse rewriting commits in your upstream. This would only
check branch X against its registered upstream. Only if there is a
registered upstream, and you're about to rewrite commits that are
reachable from the upstream remote-tracking branch, should Git
intervene and warn/refuse the rewrite. This level would IMHO provide
most of the benefit, and little or no trouble (i.e. false positives).

3. Warn/refuse rewriting _any_ 'public' commit. Refuse to rewrite any
commit that is reachable from any remote-tracking branch. Some would
say that this is a Good Thing(tm), since it prevents a commit from
being _copied_ (i.e. rebased or cherry-picked) between branches (you'd
be in this camp if you run a tightly-controlled workflow, where you
e.g. mandate upmerging patches from the oldest applicable branch
instead of cherry-picking patches from a newer branch). However, other
people would say that this is too limiting, and imposes unnecessary
rules on the workflow of the project (where e.g. copying (by way of
git-rebase) a topic branch from one place to another would cause an
annoying false positive).

[...]


Have fun! :)

...Johan

--
Johan Herland, <[hidden email]>
www.herland.net
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
On Mon, 6 Feb 2012, Johan Herland wrote:
> On Mon, Feb 6, 2012 at 15:44, Jakub Narebski <[hidden email]> wrote:
> > On Sun, 5 Feb 2012, Johan Herland wrote:

> > Relying on (default) hooks to implement this feature has the disadvantage
> > that it wouldn't be turned on by default... while this feature would be
> > most helpful for users new to git (scared by refuse to push).
>
> True. I too believe that this will be most helpful if it is enabled by
> default. That said, the easiest way to get there might be through
> first demonstrating that it works in practice when implemented as
> hooks.

Yes, starting with prototype implementation using existing infrastructure
(hooks) would be a very good idea.  (That's how first versions of what
became submodules were implemented.)

OTOH we should be aware of limitations of said prototype due to the fact
that it is a prototype...
 
> > I am not sure either if everything (wrt. safety net) can be implemented
> > via hooks.  One thing that I forgot about is preventing rewinding of
> > branch past the published commit using e.g. "git reset --hard <commit>".
> > Unless `pre-rewrite` hook could be used for that safety too...
>
> Hmm. I don't think we'll be able to "plug" all the holes that might
> leave the user in a rewritten state (e.g. what if the user (possibly
> with the help of some tool) does an "echo $SHA1 >
> .git/refs/head/master"?).

First, I was thinking about having safety net against rewriting published
commits being present only in porcelain.  Plumbing would be not affected
(perhaps there would be need to extend or add new plumbing to query "phase"
state, though).

> And trying to plug too many holes might end
> up annoying more experienced users who "know what they're doing".

Second, forcing via command line parameter should always be an option.

> Instead we might want to add a client-side check at push time. I
> realize that this check is already done by the remote end, but the
> client-side might be able to give a more helpful response along the
> lines of:

[...]

Explanation is good, but the whole idea of rewriting safety is that you
are informed (warned or denied) _before_ attempting rewrite and doing much
work.

> > > If you use 'git notes' to annotate 'public' and 'secret' states, then
> > > you can also use the --show-notes=<ref> option to let show/log display
> > > the annotations on 'public'/'secret' commits.
> >
> > First, in my opinion annotating _all_ commits with their phase is I think
> > out of question, especially annotating 'public' commits.  I don't think
> > git-notes mechanism would scale well to annotating every commit; but
> > perhaps this was tested to work, and I am mistaken.
>
> First, we don't need to annotate _all_ commits. For the 'public'
> state, we only annotate the last/tip commit that was pushed/published.
> From there, we can defer that all ancestor commits are also 'public'.

Right.

> For the 'secret' state, we do indeed annotate _all_ secret commits,
> but I believe this will be a somewhat limited number of commits. If
> your workflow forces you to annotate millions of commits as 'secret',
> I claim there is something wrong with your workflow.

Well, for the 'secret' we can rely on the fact that child of 'secret'
commit must also be 'secret' (non-publishable) if secret is to stay
secret.  Still marking all 'secret' commits might be better idea from
UI and from performance point of view.

> Second, git-notes were indeed designed scale well to handle a large
> number of notes, up to the same order of magnitude as the number of
> commits in your repo. (When git-notes was originally written, I
> successfully tested it on versions of a linux-kernel repo where every
> single commit was annotated).

Ah.  That is very nice!

> In this case, the number of 'public'
> annotations in your repo would be equal to the number of pushes you
> do, and the number of 'secret' annotations would be equal to the
> number of 'secret' commits in your repo. I'd expect both of these
> numbers to be orders of magnitude smaller than the total number of
> commits in your repo (given a fairly typical workflow in a fairly
> typical repo).

Right.
 
> > Second, I have doubts if "phase" is really state of an individual commit,
> > and not the feature of revision walking.

It matters to presentation: can commit be simultaneously 'public' because
of one branch, and 'draft' because of other.

> I believe the 'public' state is a "feature of revision walking" (i.e.
> one annotated 'public' commit implies that all its ancestors are also
> 'public'). However, the 'secret' state should be bound to the
> individual commit, IMHO.

Good call, otherwise 'secret' commit could have been "side-leaked"
by other refs being pushed.

This means though that 'public' / 'draft' while looking similar to 'secret'
are in fact a bit different things.  In other words 'immutable' and
'impushable' traits are quite a bit different in behavior...

Especially that one acts at pre-rewrite time, and second pre-push time.

> > Take for example the situation where given commit is reference by
> > remote-tracking branch 'public/foo', and also by two local branches:
> > 'foo' with upstream 'public/foo', and local branch 'bar' with no upstream.
> >
> > Now it is quite obvious that this feature should prevent rewriting 'foo'
> > branch, for which commits are published upstream.  But what about branch
> > 'bar'?  Should we prevent rewriting (e.g. rebase) here too?  What about
> > rewinding 'bar' to point somewhere else.  What if 'bar' is really detached
> > HEAD?
> >
> > These questions need to be answered...
>
> Good point. There are two questions we may need to answer: "Has commit
> X ever been published?", and "Has commit X ever been published in the
> context of branch Y?". In the latter case, we do indeed need to take
> the upstream branch into account.

I think the second one is more interested for rewrite safeties.

> Basically, there are three different "levels" for this rewrite/publish
> protection to run at:
>
> 1. Do not meddle at all. This is the current behavior, and assumes
> that if the user rewrites and pushes something, the user knows what
> he/she is doing, and Git should not meddle (obviously unless the
> server refuses the push).

I think that there should be some easy way to force such behavior,
i.e. to discard rewrite safeties.

> 2. Warn/refuse rewriting commits in your upstream. This would only
> check branch X against its registered upstream. Only if there is a
> registered upstream, and you're about to rewrite commits that are
> reachable from the upstream remote-tracking branch, should Git
> intervene and warn/refuse the rewrite. This level would IMHO provide
> most of the benefit, and little or no trouble (i.e. false positives).

Right.  I wonder if we can get usage statistics from Mercurial users
about usage of their "phases" feature... though mapping terminology
for example 'upstream' from Git to Mercurial and vice versa can be
a pain, I guess.

> 3. Warn/refuse rewriting _any_ 'public' commit. Refuse to rewrite any
> commit that is reachable from any remote-tracking branch. Some would
> say that this is a Good Thing(tm), since it prevents a commit from
> being _copied_ (i.e. rebased or cherry-picked) between branches (you'd
> be in this camp if you run a tightly-controlled workflow, where you
> e.g. mandate upmerging patches from the oldest applicable branch
> instead of cherry-picking patches from a newer branch). However, other
> people would say that this is too limiting, and imposes unnecessary
> rules on the workflow of the project (where e.g. copying (by way of
> git-rebase) a topic branch from one place to another would cause an
> annoying false positive).

Well, we could always 'deny' on 2nd, and just 'warn' on 3rd...

--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Johan Herland
On Mon, Feb 6, 2012 at 18:14, Jakub Narebski <[hidden email]> wrote:

> On Mon, 6 Feb 2012, Johan Herland wrote:
>> On Mon, Feb 6, 2012 at 15:44, Jakub Narebski <[hidden email]> wrote:
>> > Relying on (default) hooks to implement this feature has the disadvantage
>> > that it wouldn't be turned on by default... while this feature would be
>> > most helpful for users new to git (scared by refuse to push).
>>
>> True. I too believe that this will be most helpful if it is enabled by
>> default. That said, the easiest way to get there might be through
>> first demonstrating that it works in practice when implemented as
>> hooks.
>
> Yes, starting with prototype implementation using existing infrastructure
> (hooks) would be a very good idea.  (That's how first versions of what
> became submodules were implemented.)
>
> OTOH we should be aware of limitations of said prototype due to the fact
> that it is a prototype...

Agreed, but AFAICS (and modulo the addition of pre-rewrite and
pre/post-push hooks mentioned earlier) all of the things discussed so
far in this thread can be implemented as hooks.

>> > I am not sure either if everything (wrt. safety net) can be implemented
>> > via hooks.  One thing that I forgot about is preventing rewinding of
>> > branch past the published commit using e.g. "git reset --hard <commit>".
>> > Unless `pre-rewrite` hook could be used for that safety too...
>>
>> Hmm. I don't think we'll be able to "plug" all the holes that might
>> leave the user in a rewritten state (e.g. what if the user (possibly
>> with the help of some tool) does an "echo $SHA1 >
>> .git/refs/head/master"?).
>
> First, I was thinking about having safety net against rewriting published
> commits being present only in porcelain.  Plumbing would be not affected
> (perhaps there would be need to extend or add new plumbing to query "phase"
> state, though).

I also think this is very much a porcelain-only feature. I'd be more
worried if plumbing changes were needed.

>> And trying to plug too many holes might end
>> up annoying more experienced users who "know what they're doing".
>
> Second, forcing via command line parameter should always be an option.

Obviously, but if a large portion of the Git community felt the need
to always disable this feature, I'd say that we'd failed. The best
features are those that Just Work(tm).

>> Instead we might want to add a client-side check at push time. I
>> realize that this check is already done by the remote end, but the
>> client-side might be able to give a more helpful response along the
>> lines of:
>
> [...]
>
> Explanation is good, but the whole idea of rewriting safety is that you
> are informed (warned or denied) _before_ attempting rewrite and doing much
> work.

True, but there may be cases where the rewrite is not apparent until
after it has happened. E.g. a novice user may use 'git reset --hard'
in order to get to an earlier state for testing purposes, and then -
after completing the test - 'git reset --hard' back to the starting
point. I know this is not best practice, but is it bad enough that we
want to refuse it?

>> First, we don't need to annotate _all_ commits. For the 'public'
>> state, we only annotate the last/tip commit that was pushed/published.
>> From there, we can defer that all ancestor commits are also 'public'.
>
> Right.
>
>> For the 'secret' state, we do indeed annotate _all_ secret commits,
>> but I believe this will be a somewhat limited number of commits. If
>> your workflow forces you to annotate millions of commits as 'secret',
>> I claim there is something wrong with your workflow.
>
> Well, for the 'secret' we can rely on the fact that child of 'secret'
> commit must also be 'secret' (non-publishable) if secret is to stay
> secret.  Still marking all 'secret' commits might be better idea from
> UI and from performance point of view.

I don't think we should automatically assume that all children of a
'secret' commit are also 'secret'. First of all, the git DAG was not
made for iterating forwards in history, so given a 'secret' commit, it
is computationally expensive to enumerate all its implied-'secret'
descendants. More importantly though, I don't agree with the premise.
I would typically use the 'secret' state as follows: While debugging a
piece of code, I might commit a few debug print statements, and I
would typically mark this debug commit as 'secret', in order to
prevent myself from accidentally pushing this. Although it probably
doesn't matter in practice, I think it is wrong for the commits made
(temporarily) on top of this debug commit to be also considered
'secret'. They are only unpublishable as a consequence of being based
on the debug commit, and only until I get around to rebasing away the
'secret' debug commit.

>> > Second, I have doubts if "phase" is really state of an individual commit,
>> > and not the feature of revision walking.
>
> It matters to presentation: can commit be simultaneously 'public' because
> of one branch, and 'draft' because of other.
>
>> I believe the 'public' state is a "feature of revision walking" (i.e.
>> one annotated 'public' commit implies that all its ancestors are also
>> 'public'). However, the 'secret' state should be bound to the
>> individual commit, IMHO.
>
> Good call, otherwise 'secret' commit could have been "side-leaked"
> by other refs being pushed.
>
> This means though that 'public' / 'draft' while looking similar to 'secret'
> are in fact a bit different things.  In other words 'immutable' and
> 'impushable' traits are quite a bit different in behavior...
>
> Especially that one acts at pre-rewrite time, and second pre-push time.

Exactly. I find Mercurial 'phase' language confusing, precisely for
the reason that 'public' and 'secret' are DIFFERENT concepts. One
hinders rewrite and naturally applies to a commit AND its ancestors,
while the other hinders push and only applies to the commit itself.
The fact that they could be implemented by the same mechanisms (hooks
and notes) does not make them the same thing.

>> > Take for example the situation where given commit is reference by
>> > remote-tracking branch 'public/foo', and also by two local branches:
>> > 'foo' with upstream 'public/foo', and local branch 'bar' with no upstream.
>> >
>> > Now it is quite obvious that this feature should prevent rewriting 'foo'
>> > branch, for which commits are published upstream.  But what about branch
>> > 'bar'?  Should we prevent rewriting (e.g. rebase) here too?  What about
>> > rewinding 'bar' to point somewhere else.  What if 'bar' is really detached
>> > HEAD?
>> >
>> > These questions need to be answered...
>>
>> Good point. There are two questions we may need to answer: "Has commit
>> X ever been published?", and "Has commit X ever been published in the
>> context of branch Y?". In the latter case, we do indeed need to take
>> the upstream branch into account.
>
> I think the second one is more interested for rewrite safeties.
>
>> Basically, there are three different "levels" for this rewrite/publish
>> protection to run at:
>>
>> 1. Do not meddle at all. This is the current behavior, and assumes
>> that if the user rewrites and pushes something, the user knows what
>> he/she is doing, and Git should not meddle (obviously unless the
>> server refuses the push).
>
> I think that there should be some easy way to force such behavior,
> i.e. to discard rewrite safeties.

Indeed. We should probably have a simple config flag to enable the
rewrite protection. In fact I would argue that the flag should default
to false (disable protection) when unset, and then we should let
init/clone set the config flag to true (enable protection) in newly
created repos (unless explicitly disabled in the user/system configs).
This way, behavior does not change for existing repos, but new repos
are protected by default (with only a single command needed to disable
the protection).

>> 2. Warn/refuse rewriting commits in your upstream. This would only
>> check branch X against its registered upstream. Only if there is a
>> registered upstream, and you're about to rewrite commits that are
>> reachable from the upstream remote-tracking branch, should Git
>> intervene and warn/refuse the rewrite. This level would IMHO provide
>> most of the benefit, and little or no trouble (i.e. false positives).
>
> Right.  I wonder if we can get usage statistics from Mercurial users
> about usage of their "phases" feature... though mapping terminology
> for example 'upstream' from Git to Mercurial and vice versa can be
> a pain, I guess.

I'm unsure how useful it would be. IMHO Git and Mercurial are
different enough (and promote sightly different workflows) that I
don't trust the the average Mercurial user's preference for
Mercurial's 'phase' behavior to be transferable to the average Git
user's preference for a similar behavior in Git.

>> 3. Warn/refuse rewriting _any_ 'public' commit. Refuse to rewrite any
>> commit that is reachable from any remote-tracking branch. Some would
>> say that this is a Good Thing(tm), since it prevents a commit from
>> being _copied_ (i.e. rebased or cherry-picked) between branches (you'd
>> be in this camp if you run a tightly-controlled workflow, where you
>> e.g. mandate upmerging patches from the oldest applicable branch
>> instead of cherry-picking patches from a newer branch). However, other
>> people would say that this is too limiting, and imposes unnecessary
>> rules on the workflow of the project (where e.g. copying (by way of
>> git-rebase) a topic branch from one place to another would cause an
>> annoying false positive).
>
> Well, we could always 'deny' on 2nd, and just 'warn' on 3rd...

Good idea.


Have fun! :)

...Johan

--
Johan Herland, <[hidden email]>
www.herland.net
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
On Mon, 6 Feb 2012, Johan Herland wrote:
> On Mon, Feb 6, 2012 at 18:14, Jakub Narebski <[hidden email]> wrote:
>> On Mon, 6 Feb 2012, Johan Herland wrote:
>>> On Mon, Feb 6, 2012 at 15:44, Jakub Narebski <[hidden email]> wrote:

[...]

>> Yes, starting with prototype implementation using existing infrastructure
>> (hooks) would be a very good idea.  (That's how first versions of what
>> became submodules were implemented.)
>>
>> OTOH we should be aware of limitations of said prototype due to the fact
>> that it is a prototype...
>
> Agreed, but AFAICS (and modulo the addition of pre-rewrite and
> pre/post-push hooks mentioned earlier) all of the things discussed so
> far in this thread can be implemented as hooks.

That would be nice.

And the new hooks (pre-rewrite, pre/post-push) would be useful anyway,
I think.

[...]
>>> And trying to plug too many holes might end
>>> up annoying more experienced users who "know what they're doing".
>>
>> Second, forcing via command line parameter should always be an option.
>
> Obviously, but if a large portion of the Git community felt the need
> to always disable this feature, I'd say that we'd failed. The best
> features are those that Just Work(tm).

Well, we have some features in git like advice.* that cater to newbies.
This could be such newbie-mostly feature too.

Anyway what I was thinking about is that forcing is for those hopefully
rare cases where Git safety net is too strict, where DWIM-mery fails.

>>> Instead we might want to add a client-side check at push time. I
>>> realize that this check is already done by the remote end, but the
>>> client-side might be able to give a more helpful response along the
>>> lines of:
>>
>> [...]
>>
>> Explanation is good, but the whole idea of rewriting safety is that you
>> are informed (warned or denied) _before_ attempting rewrite and doing much
>> work.
>
> True, but there may be cases where the rewrite is not apparent until
> after it has happened. E.g. a novice user may use 'git reset --hard'
> in order to get to an earlier state for testing purposes, and then -
> after completing the test - 'git reset --hard' back to the starting
> point. I know this is not best practice, but is it bad enough that we
> want to refuse it?

I'd like to have safety net also for using 'git reset --hard' to rewind
publishable branch past published history.  Though preventing e.g. force
rename of other branch to publishable branch so it doesn't fast-forward
WRT its upstream would be going to far, I guess...
 

>>> First, we don't need to annotate _all_ commits. For the 'public'
>>> state, we only annotate the last/tip commit that was pushed/published.
>>> From there, we can defer that all ancestor commits are also 'public'.
>>
>> Right.
>>
>>> For the 'secret' state, we do indeed annotate _all_ secret commits,
>>> but I believe this will be a somewhat limited number of commits. If
>>> your workflow forces you to annotate millions of commits as 'secret',
>>> I claim there is something wrong with your workflow.
>>
>> Well, for the 'secret' we can rely on the fact that child of 'secret'
>> commit must also be 'secret' (non-publishable) if secret is to stay
>> secret.  Still marking all 'secret' commits might be better idea from
>> UI and from performance point of view.
>
> I don't think we should automatically assume that all children of a
> 'secret' commit are also 'secret'.

All right.  What is important is that safety net related to 'secret'
commits would prevent publishing such commits even if you build non-secret
commits on top of 'secret' one.

>                                 First of all, the git DAG was not
> made for iterating forwards in history, so given a 'secret' commit, it
> is computationally expensive to enumerate all its implied-'secret'
> descendants.

Right, this would enormously complicate detecting 'secret'-ness of commit
when browsing history / querying "phase" of a commit..

> More importantly though, I don't agree with the premise.
> I would typically use the 'secret' state as follows: While debugging a
> piece of code, I might commit a few debug print statements, and I
> would typically mark this debug commit as 'secret', in order to
> prevent myself from accidentally pushing this. Although it probably
> doesn't matter in practice, I think it is wrong for the commits made
> (temporarily) on top of this debug commit to be also considered
> 'secret'. They are only unpublishable as a consequence of being based
> on the debug commit, and only until I get around to rebasing away the
> 'secret' debug commit.

Right, thanks for sanity check.

So another difference that in its full complexity the 'secret' is an
attribute of an individual commit which is (usually) local to repository,
while 'public' is in its full complexity an attribute of revision walking
rather than something that can be attached to a commit.

Also, when thinking about different scenarios of why one would like to
mark commit as 'secret', we might want to be able to mark commit as
secret / unpublishable with respect to _subset_ of remotes, so e.g.
I am prevented from accidentally publishing commits marked as 'secret'
to public repository, or to CI/QA repository, but I can push (perhaps
with warning) to group repository, together with 'secret'-ness state
of said commit...

... though it wouldn't be as much 'secret' as 'confidential' ;-)

>>>> Second, I have doubts if "phase" is really state of an individual commit,
>>>> and not the feature of revision walking.
>>
>> It matters to presentation: can commit be simultaneously 'public' because
>> of one branch, and 'draft' because of other.
>>
>>> I believe the 'public' state is a "feature of revision walking" (i.e.
>>> one annotated 'public' commit implies that all its ancestors are also
>>> 'public'). However, the 'secret' state should be bound to the
>>> individual commit, IMHO.
>>
>> Good call, otherwise 'secret' commit could have been "side-leaked"
>> by other refs being pushed.
>>
>> This means though that 'public' / 'draft' while looking similar to 'secret'
>> are in fact a bit different things.  In other words 'immutable' and
>> 'impushable' traits are quite a bit different in behavior...
>>
>> Especially that one acts at pre-rewrite time, and second pre-push time.
>
> Exactly. I find Mercurial 'phase' language confusing, precisely for
> the reason that 'public' and 'secret' are DIFFERENT concepts. One
> hinders rewrite and naturally applies to a commit AND its ancestors,
> while the other hinders push and only applies to the commit itself.
> The fact that they could be implemented by the same mechanisms (hooks
> and notes) does not make them the same thing.

Well, if one doesn't think about all possible usages and complications,
the simplicity of metaphor of "phases" is quite compelling.  We have
(at first glance)

   public < draft < secret

states of a commit (changeset), just like we have

   solid < liquid < gaseous

phases of matter.  

If we treat traits of 'immutability' and 'publishability' as true boolean,
such ordering of "phases" looks natural.  Just like matter goes from gas
to liquid to solid with lowering temperature, changeset "phase" goes from
'secret' to 'draft' to 'public' following ancestry chain.

But if we go for expressivity and power like the rest of Git, rather than
for simplicity like Mercurial does, the differences between 'secret'-ness
and 'public'-ness make it unlikely that it would be a good idea to encompass
those in a single UI and a single concept.  I guess that it can be another
issue where Mercurial oversimplifies its approach (c.f. branches which
started from clone to branch and anonymous branches, to "named branches"
which are really commit labels, to local-only bookmark branches, to
transferrable Git-like bookmark branches but with single namespace, and
only just considering equivalent of Git's refspec and refs mapping; c.f.
revision ranges based on local numbering of commits which made ranges
local too, to new Git-like topological `--ancestry-path`-like ranges).


Also with per-remote 'secret'-ness, we might have the scenario where
a commit is marked as 'secret' for public repo, we have pushed it to
group repo (and it's 'secret'-ness trait together with it), an now we
want to be warned if we try to rewrite said commit.

So, separate traits it is (of revision walk, or of a commit), rather
than "smush-together" antity / attribute of "phase" of revision.


That is BTW why I wanted us to have this discussion... :-D

[...]

>>> Basically, there are three different "levels" for this rewrite/publish
>>> protection to run at:
>>>
>>> 1. Do not meddle at all. This is the current behavior, and assumes
>>> that if the user rewrites and pushes something, the user knows what
>>> he/she is doing, and Git should not meddle (obviously unless the
>>> server refuses the push).
>>
>> I think that there should be some easy way to force such behavior,
>> i.e. to discard rewrite safeties.
>
> Indeed. We should probably have a simple config flag to enable the
> rewrite protection. In fact I would argue that the flag should default
> to false (disable protection) when unset, and then we should let
> init/clone set the config flag to true (enable protection) in newly
> created repos (unless explicitly disabled in the user/system configs).
> This way, behavior does not change for existing repos, but new repos
> are protected by default (with only a single command needed to disable
> the protection).

And commit option for discarding safeties for specific push, or for
specific rewrite (e.g. pushed at night, noticed an error in commit
message, force-amended, and force-pushed again, hoping that the window
of opportunity is sufficintly small so that nobody will be affected).

>>> 2. Warn/refuse rewriting commits in your upstream. This would only
>>> check branch X against its registered upstream. Only if there is a
>>> registered upstream, and you're about to rewrite commits that are
>>> reachable from the upstream remote-tracking branch, should Git
>>> intervene and warn/refuse the rewrite. This level would IMHO provide
>>> most of the benefit, and little or no trouble (i.e. false positives).
>>
>> Right.  I wonder if we can get usage statistics from Mercurial users
>> about usage of their "phases" feature... though mapping terminology
>> for example 'upstream' from Git to Mercurial and vice versa can be
>> a pain, I guess.
>
> I'm unsure how useful it would be. IMHO Git and Mercurial are
> different enough (and promote sightly different workflows) that I
> don't trust the the average Mercurial user's preference for
> Mercurial's 'phase' behavior to be transferable to the average Git
> user's preference for a similar behavior in Git.

Well, bad statistics might be better than no statistics.  What would
be especially interesting is _complaints_ about "phases" feature,
don't you think, i.e. where the "phases" metaphor fails.

[...]

--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Johan Herland
(we are pretty much in violent agreement, so I will only comment where
I find it necessary)

On Tue, Feb 7, 2012 at 15:31, Jakub Narebski <[hidden email]> wrote:
> Also, when thinking about different scenarios of why one would like to
> mark commit as 'secret', we might want to be able to mark commit as
> secret / unpublishable with respect to _subset_ of remotes, so e.g.
> I am prevented from accidentally publishing commits marked as 'secret'
> to public repository, or to CI/QA repository, but I can push (perhaps
> with warning) to group repository, together with 'secret'-ness state
> of said commit...
>
> ... though it wouldn't be as much 'secret' as 'confidential' ;-)

Another way to achieve this would be to have a config flag to control
whether Git checks for the 'secret' flag before pushing. This config
flag could be set at the system/user level (to enable/disable the
feature as a whole), at the repo level (to enable/disable it in a
given repo), at the remote level (to enable/disable it on a given
repo), and finally at the branch level (to enable-disable it for a
given branch (and its upstream)). Thus you could have a .git/config
that looked like this:

  [core]
  refusePushSecret = true

  [remote "foo"]
  refusePushSecret = false
  url = ...
  fetch = ...

  [branch "baz"]
  remote = foo
  merge = refs/heads/baz
  refusePushSecret = true

This config would:

 - refuse to push 'secret' commits from branch 'baz'
(branch.baz.refusePushSecret == true)

 - but allow to push other branches with 'secret' commits to remote
'foo' (remote.foo.refusePushSecret == false)

 - but refuse to push 'secret' commits to other remotes
(core.refusePushSecret == true)

(The order of precedence would be: branch config > remote config >
repo config > user config > system config > default when unset)

I am unsure whether the 'secret'-ness of a commit should follow across
the push, but if you do (assuming we store the 'secret' flag using
git-notes) this is simply a matter of synchronizing the
refs/notes/secret to the same remote.


Have fun! :)

...Johan

--
Johan Herland, <[hidden email]>
www.herland.net
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Ronan Keryell
In reply to this post by Jakub Narębski
>>>>> On Tue, 7 Feb 2012 15:31:07 +0100, Jakub Narebski <[hidden email]> said:

    >> Agreed, but AFAICS (and modulo the addition of pre-rewrite and
    >> pre/post-push hooks mentioned earlier) all of the things
    >> discussed so far in this thread can be implemented as hooks.

    Jakub> That would be nice.

    Jakub> And the new hooks (pre-rewrite, pre/post-push) would be
    Jakub> useful anyway, I think.

Yes, to deal with file metadata in git for example. :-)
--
  Ronan KERYELL                      |\/  Phone:  +1 408 844 4720
  Wild Systems / HPC Project, Inc.   |/)  Cell:   +33 613 143 766
  5201 Great America Parkway #3241   K    [hidden email]
  Santa Clara, CA 95054              |\   skype:keryell
  USA                                | \  http://hpc-project.com
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Jakub Narębski
In reply to this post by Johan Herland
On Tue, 7 Feb 2012, Johan Herland wrote:

> (we are pretty much in violent agreement, so I will only comment where
> I find it necessary)

So now comes the hard part: actually implementing (well, designing and
implementing) prototypes for 'secret' trait and 'public' trait...
 
> On Tue, Feb 7, 2012 at 15:31, Jakub Narebski <[hidden email]> wrote:

> > Also, when thinking about different scenarios of why one would like to
> > mark commit as 'secret', we might want to be able to mark commit as
> > secret / unpublishable with respect to _subset_ of remotes, so e.g.
> > I am prevented from accidentally publishing commits marked as 'secret'
> > to public repository, or to CI/QA repository, but I can push (perhaps
> > with warning) to group repository, together with 'secret'-ness state
> > of said commit...
> >
> > ... though it wouldn't be as much 'secret' as 'confidential' ;-)
>
> Another way to achieve this would be to have a config flag to control
> whether Git checks for the 'secret' flag before pushing. This config
> flag could be set at the system/user level (to enable/disable the
> feature as a whole), at the repo level (to enable/disable it in a
> given repo), at the remote level (to enable/disable it on a given
> repo), and finally at the branch level (to enable-disable it for a
> given branch (and its upstream)). Thus you could have a .git/config
> that looked like this:
>
>   [core]
>   refusePushSecret = true
>
>   [remote "foo"]
>   refusePushSecret = false
>   url = ...
>   fetch = ...
>
>   [branch "baz"]
>   remote = foo
>   merge = refs/heads/baz
>   refusePushSecret = true
>
> This config would:
>
>  - refuse to push 'secret' commits from branch 'baz'
> (branch.baz.refusePushSecret == true)
>
>  - but allow to push other branches with 'secret' commits to remote
> 'foo' (remote.foo.refusePushSecret == false)
>
>  - but refuse to push 'secret' commits to other remotes
> (core.refusePushSecret == true)
>
> (The order of precedence would be: branch config > remote config >
> repo config > user config > system config > default when unset)

You read my mind.

> I am unsure whether the 'secret'-ness of a commit should follow across
> the push, but if you do (assuming we store the 'secret' flag using
> git-notes) this is simply a matter of synchronizing the
> refs/notes/secret to the same remote.

I think it should, so that 'secret' commit would not escape by accident
via a group secret repository.

What makes it hard (I think) is that we would prefer to transfer
'secret'-ness only for pushed commits.  That might be problem for notes
based implementation of 'secret' annotation and 'secret'-ness transfer...
though I guess knowing that there exist 'secret' commit with given SHA1
which we do not have and should not have is not much breach of
confidentiality.  Still...

--
Jakub Narebski
Poland
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Philip Oakley
From: "Jakub Narebski" <[hidden email]> Sent: Friday, February 10, 2012
7:38 PM
> On Tue, 7 Feb 2012, Johan Herland wrote:
>
>> (we are pretty much in violent agreement, so I will only comment where
>> I find it necessary)
>
> So now comes the hard part: actually implementing (well, designing and
> implementing) prototypes for 'secret' trait and 'public' trait...

It all sounds very sensible.

The one additional implemenation idea I'd like is to have, which is somewhat
at a tangent to the secrecy issue, is having some record of what was pushed
(made Public). For the sub-module case this is so that its super project can
know if its submodules are public or not. A public super project that has
'secret' sub-modules is awkward to say the least.

That's my thought anyway...

>
>> On Tue, Feb 7, 2012 at 15:31, Jakub Narebski <[hidden email]> wrote:
>
>> > Also, when thinking about different scenarios of why one would like to
>> > mark commit as 'secret', we might want to be able to mark commit as
>> > secret / unpublishable with respect to _subset_ of remotes, so e.g.
>> > I am prevented from accidentally publishing commits marked as 'secret'
>> > to public repository, or to CI/QA repository, but I can push (perhaps
>> > with warning) to group repository, together with 'secret'-ness state
>> > of said commit...
>> >
>> > ... though it wouldn't be as much 'secret' as 'confidential' ;-)
>>
>> Another way to achieve this would be to have a config flag to control
>> whether Git checks for the 'secret' flag before pushing. This config
>> flag could be set at the system/user level (to enable/disable the
>> feature as a whole), at the repo level (to enable/disable it in a
>> given repo), at the remote level (to enable/disable it on a given
>> repo), and finally at the branch level (to enable-disable it for a
>> given branch (and its upstream)). Thus you could have a .git/config
>> that looked like this:
>>
>>   [core]
>>   refusePushSecret = true
>>
>>   [remote "foo"]
>>   refusePushSecret = false
>>   url = ...
>>   fetch = ...
>>
>>   [branch "baz"]
>>   remote = foo
>>   merge = refs/heads/baz
>>   refusePushSecret = true
>>
>> This config would:
>>
>>  - refuse to push 'secret' commits from branch 'baz'
>> (branch.baz.refusePushSecret == true)
>>
>>  - but allow to push other branches with 'secret' commits to remote
>> 'foo' (remote.foo.refusePushSecret == false)
>>
>>  - but refuse to push 'secret' commits to other remotes
>> (core.refusePushSecret == true)
>>
>> (The order of precedence would be: branch config > remote config >
>> repo config > user config > system config > default when unset)
>
> You read my mind.
>
>> I am unsure whether the 'secret'-ness of a commit should follow across
>> the push, but if you do (assuming we store the 'secret' flag using
>> git-notes) this is simply a matter of synchronizing the
>> refs/notes/secret to the same remote.
>
> I think it should, so that 'secret' commit would not escape by accident
> via a group secret repository.
>
> What makes it hard (I think) is that we would prefer to transfer
> 'secret'-ness only for pushed commits.  That might be problem for notes
> based implementation of 'secret' annotation and 'secret'-ness transfer...
> though I guess knowing that there exist 'secret' commit with given SHA1
> which we do not have and should not have is not much breach of
> confidentiality.  Still...
>
> --
> Jakub Narebski
> Poland
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to [hidden email]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2012.0.1913 / Virus Database: 2112/4801 - Release Date: 02/10/12
>

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [RFD] Rewriting safety - warn before/when rewriting published history

Johan Herland
In reply to this post by Jakub Narębski
On Fri, Feb 10, 2012 at 20:38, Jakub Narebski <[hidden email]> wrote:

> On Tue, 7 Feb 2012, Johan Herland wrote:
>> On Tue, Feb 7, 2012 at 15:31, Jakub Narebski <[hidden email]> wrote:
>> I am unsure whether the 'secret'-ness of a commit should follow across
>> the push, but if you do (assuming we store the 'secret' flag using
>> git-notes) this is simply a matter of synchronizing the
>> refs/notes/secret to the same remote.
>
> I think it should, so that 'secret' commit would not escape by accident
> via a group secret repository.
>
> What makes it hard (I think) is that we would prefer to transfer
> 'secret'-ness only for pushed commits.  That might be problem for notes
> based implementation of 'secret' annotation and 'secret'-ness transfer...
> though I guess knowing that there exist 'secret' commit with given SHA1
> which we do not have and should not have is not much breach of
> confidentiality.  Still...

If you don't want to transfer all of refs/notes/secret, you would
probably have to extend the git protocol with a per-commit 'secret'
flag (which would then be applied to the receiving repo's
refs/notes/secret).

Still, this is all specific to the 'secret' feature, which IMHO is
much less important then the 'public' feature. Implementing the
barebones 'public' feature (i.e. refuse rewrite of commits reachable
from upstream) is much less work, and would be enough for 90% of git
users, I believe.


...Johan

--
Johan Herland, <[hidden email]>
www.herland.net
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
12
Loading...