[PATCHv8 0/5] pathspec magic extension to search for attributes

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: [PATCHv8 0/5] pathspec magic extension to search for attributes

Stefan Beller-4
On Sun, May 22, 2016 at 4:45 AM, Duy Nguyen <[hidden email]> wrote:
> On Fri, May 20, 2016 at 4:25 AM, Stefan Beller <[hidden email]> wrote:
>>> My take is to pretend sparse checkout does not exist at all and then
>>> go from there ;-)
>
> Hehe.. shameless plug, narrow checkout [1] should be its great
> successor where everything is done right (famous last words). Maybe I
> can convince Stefan to finish that off then I'll finally bring narrow
> clone!

I started reviewing that (see separate email).

So conceptually the narrow checkout is "sparse checkout just for
directories" IIUC.

A submodule is treated as a files (sometimes, e.g. diff) and sometimes
as a directory (e.g. in the working tree)

>
> [1] http://article.gmane.org/gmane.comp.version-control.git/289112
>
>> Using these pathspec attrs are a good example for why we would
>> want to make it more generic. Imagine a future, where more attributes
>> can be codified into pathspecs and this is one of the possibilities:
>>
>>     git clone --sparse=":(exclude,size>5MB,binary)
>>
>> which would not checkout files that are large binary files. We could
>> also extend the protocol (v2 with the capabilities in client speaks first)
>> to transmit such a requirement to the server.
>
> I think you need narrow clone there ;-) It's the first step to have a
> clone with missing directories. I think then we can extend it further
> to exclude (large) files.

But if we only exclude some files we are not having a binary
decision for a directory, so narrow checkout doesn't work here?

    git clone --sparse=":(exclude,size>5MB,binary)

would maybe lead to have
  dir/path.c #included
  dir/binary-asset.img # excluded

so it is more a sparse thing than a narrow thing?

>
>> Why is sparseness considered bad?
>
> It does not scale well when the worktree gets bigger. It can be slow
> (but this is just a technical problem I haven't spent time on fixing).
> And it does not enable narrow clone (not with lots of hacks, I think I
> did just that in some early iterations).
>
>> If I wanted to just do the submodule only thing, this would be a smaller
>> series, I would guess.
>
> No no no. Do more. Start with narrow checkout. I'll help ;-)

Thanks for the encouragement!
How is the interface going to look like for the narrow checkout/clone ?
If the UI is supposed to be very similar, we can merge the two concepts
and make it one thing.

But if the UI is different enough, we may want to keep it separate as
it is not as confusing to the user as we'd think?

What I imagine UI-wise, see
http://thread.gmane.org/gmane.comp.version-control.git/295221


Thanks,
Stefan

> --
> Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Reply | Threaded
Open this post in threaded view
|

Re: [PATCHv8 0/5] pathspec magic extension to search for attributes

Duy Nguyen
On Tue, May 24, 2016 at 1:49 AM, Stefan Beller <[hidden email]> wrote:

> On Sun, May 22, 2016 at 4:45 AM, Duy Nguyen <[hidden email]> wrote:
>> On Fri, May 20, 2016 at 4:25 AM, Stefan Beller <[hidden email]> wrote:
>>>> My take is to pretend sparse checkout does not exist at all and then
>>>> go from there ;-)
>>
>> Hehe.. shameless plug, narrow checkout [1] should be its great
>> successor where everything is done right (famous last words). Maybe I
>> can convince Stefan to finish that off then I'll finally bring narrow
>> clone!
>
> I started reviewing that (see separate email).
>
> So conceptually the narrow checkout is "sparse checkout just for
> directories" IIUC.

Yep.

> A submodule is treated as a files (sometimes, e.g. diff) and sometimes
> as a directory (e.g. in the working tree)

It's the same here. Submodules have a special mode in the index
(S_IFDIR + S_IF<something>) while dirs just have S_IFDIR. It's up to
the index user to treat it as a "file" or a directory (maybe only
unpack-trees)

>>> Using these pathspec attrs are a good example for why we would
>>> want to make it more generic. Imagine a future, where more attributes
>>> can be codified into pathspecs and this is one of the possibilities:
>>>
>>>     git clone --sparse=":(exclude,size>5MB,binary)
>>>
>>> which would not checkout files that are large binary files. We could
>>> also extend the protocol (v2 with the capabilities in client speaks first)
>>> to transmit such a requirement to the server.
>>
>> I think you need narrow clone there ;-) It's the first step to have a
>> clone with missing directories. I think then we can extend it further
>> to exclude (large) files.
>
> But if we only exclude some files we are not having a binary
> decision for a directory, so narrow checkout doesn't work here?
>
>     git clone --sparse=":(exclude,size>5MB,binary)
>
> would maybe lead to have
>   dir/path.c #included
>   dir/binary-asset.img # excluded
>
> so it is more a sparse thing than a narrow thing?

I wrote two paragraphs, only to realized that you were right. In the
end we just need a way to exchange "sparse" (not narrow) information
about this. Narrow clone may need to exchange more side information
(for git-replace) but I don't see how sparse files fit exactly in this
whole exchange yet.

>>> Why is sparseness considered bad?
>>
>> It does not scale well when the worktree gets bigger. It can be slow
>> (but this is just a technical problem I haven't spent time on fixing).
>> And it does not enable narrow clone (not with lots of hacks, I think I
>> did just that in some early iterations).
>>
>>> If I wanted to just do the submodule only thing, this would be a smaller
>>> series, I would guess.
>>
>> No no no. Do more. Start with narrow checkout. I'll help ;-)
>
> Thanks for the encouragement!
> How is the interface going to look like for the narrow checkout/clone ?
> If the UI is supposed to be very similar, we can merge the two concepts
> and make it one thing.
>
> But if the UI is different enough, we may want to keep it separate as
> it is not as confusing to the user as we'd think?

For narrow clone, I imagine you need to specify what directories, say
with --narrow-path option or simply --path, to clone, and that's it.
In future you may add --path-filter option that does
"exclude,size>5MB,binary" above.

For narrow checkout, we have two more operations: fold and unfold
directories. At plumbing level, maybe update-index can support these
operations explicitly. At porcelain level, checkout probably learns
about --narrow-path and submodule can make use of it if it wants.

> What I imagine UI-wise, see
> http://thread.gmane.org/gmane.comp.version-control.git/295221
>
>
> Thanks,
> Stefan
>
>> --
>> Duy



--
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to [hidden email]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
12