What will happen to ZFS Crypto?

Added by Paul Harper over 3 years ago

What will happen to ZFS Crypto? Is it 'usable'?


Replies (19)

RE: What will happen to ZFS Crypto? - Added by Jonathan Edwards over 3 years ago

is Darren Moffat still there, or has he defected yet? .. he was the primary driver behind the effort IIRC

RE: What will happen to ZFS Crypto? - Added by Charles Morris over 3 years ago

The ZFS Crypto project is one of the most important goals, in my opinion. It still needs a ton of work if I recall, however personally I think it's worth the effort.

RE: What will happen to ZFS Crypto? - Added by Sargon of AKKAD over 3 years ago

I have reason to believe that ZFS crypto will be (is?) available in SNV_149 and Solaris 11 / Solaris 11 Express

I think this based on the references below, and anyone can argue an alternative interpretation:

"OK, I'll spend more time this week discussing many of the other innovations we're planning, around Security for sure (ZFS crypto anyone?)"
http://blogs.oracle.com/solaris/2010/09/oracle_solaris_11_outlined.html

"Bug ID 4854202
Synopsis ZFS data set encryption
State 10-Fix Delivered (Fix available in build)
Fixed In snv_149
Release Fixed solaris_nevada(snv_149) "
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=4854202

If it is indeed in SNV_149 it should be no trouble to move it to Illumos. If it has been restricted to Solaris 11, it is not clear when/if we will have access to the source.

RE: What will happen to ZFS Crypto? - Added by Paul Harper over 3 years ago

The current state of play for ZFS Crypto.

RE: What will happen to ZFS Crypto? - Added by Charles Morris over 3 years ago

Whatever we end up doing, we should make sure that periodic key inversion is supported; in order to protect against RAM bias. I'm not certain the current Solaris 11 Express implementation does (it is trivial to implement, however). There are many such requirements that are not in the original OpenSolaris zfs-crypto project requirements documentation. Does anybody even know if the current release and the original spec are even close to matching? Knowing Oracle's information disclosure policy, I'm guessing not.

If anyone decides to take a crack at illumos-zfs-crypto please keep these boards up to date on the topics, so I know who to assist.

RE: What will happen to ZFS Crypto? - Added by Mr. Hotzen over 2 years ago

Are there any news on the ZFS crypto issu?
Is the ZFS-source to be expected with the Solaris 11 (Non-Express) Release and going to be integrated?

That is the #1 feature keeping me from switching to OI :(

RE: What will happen to ZFS Crypto? - Added by Sargon of AKKAD over 2 years ago

I haven't heard anything interesting for a year. I believe Solaris 11 is expected to be released in November.

The leaked memo from August of 2010 said:
"We will continue to use the CDDL license statement in nearly all Solaris source code files. We will not remove the CDDL from any files in Solaris to which it already applies, and new source code files that are created will follow the current policy regarding applying the CDDL (simply, that usr/src files will have the CDDL, and the very small minority of files in usr/closed might not have it). Use of other open licenses in non-ON consolidations (e.g. GPL in the Desktop area) will also continue. As before, requests to change the license associated with source code are case-by-case decisions.

We will distribute updates to approved CDDL or other open source- licensed code following full releases of our enterprise Solaris operating system. In this manner, new technology innovations will
show up in our releases before anywhere else. We will no longer distribute source code for the entirety of the Solaris operating system in real-time while it is developed, on a nightly basis."

If it is still the case that the source for ZFS crypto will be release after the full operating system, Darren Moffatt will take his rightful place on my list of personal heroes. Otherwise, I will cry for a long, long time.

RE: What will happen to ZFS Crypto? - Added by Paul Harper over 2 years ago

Did Oracle release the source code? Will ZFS-Crypto be on the way into Illumos?

RE: What will happen to ZFS Crypto? - Added by Sargon of AKKAD over 2 years ago

Good question. The old ZFS Crypto Solaris page is gone: http://hub.opensolaris.org/bin/view/Project%2Bzfs-crypto/

but this site now exists: https://www.illumos.org/projects/zfs-crypto

What does it mean?

RE: What will happen to ZFS Crypto? - Added by Linda Kateley over 2 years ago

We are not sure of what oracle is doing and we function independently of them.

I am aware of a couple groups who would like to see zfs crypto in illumos so a project was added to encourage this open development.

RE: What will happen to ZFS Crypto? - Added by Sargon of AKKAD over 2 years ago

So here's what I know, the OpenSolaris site, http://hub.opensolaris.org/bin/view/Project+zfs-crypto/ is back up, with a few new links.
Solaris 11 is officially released: http://www.oracle.com/technetwork/server-storage/solaris11/overview/index.html
I still don't know anything about the source code.

This from Darren Moffat in July: "I do not know anything about future plans nor would I be able to comment here at this time even if I did. Please bring this up with your Oracle account/support team representative if it is important to your business. Is there something in particular you want to do with the source if you had it available to you ? Are there changes you want to make ?"

RE: What will happen to ZFS Crypto? - Added by Linda Kateley over 2 years ago

One of the problems we have as a community is that there are many members of the community who were former sun/oracle kernel engineers and may have seen the oracle source. They may have legal problems in contributing to an open crypto. There is a project that has been started and there are a number of contributors who are not legacy sun/oracle who are working on the project. Even if oracle published the source.. alot of decisions would have to made on whether to use it. Creating our own illumos solution is probably the best idea.

RE: What will happen to ZFS Crypto? - Added by Jorgen Lundman over 2 years ago

With the unintentional present from Oracle, and the sources to zpool version 33 which includes zfs-crypto. I guess now more than ever there needs to be a discussion on how to proceed.

But I guess it is anyone's guess as to the license of these files. They all have CDDL headers on all the ZFS files, but weren't exactly officially released... but internally they were already tagged CDDL...

RE: What will happen to ZFS Crypto? - Added by Charles Morris over 2 years ago

I agree with Linda, continuing the independent development of zfs-crypto and bp_rewrite is the best solution even considering the code release.

Considering all the lies Oracle told relating to OpenSolaris and, well, everything in general; even if they have handed us this code on a platter no-strings-attached, we absolutely cannot expect code updates, roadmaps, or etc. Blindly including the tree from oracle could lead to a bad situation. Illumos code will be better anyway.

I have the understanding that the theoretical approach outlined in the prior-takeover times is generally the same as the one taken by the 'oracle' code in the release. This should only strengthen the approach and should not, in my opinion, restrict development along these lines.

The fact that oracle tagged the files with CDDL certainly implies... something, but I am going to query a lawyer about the possible copyright issues here.

n.b. don't listen to me, talk to lawyer

RE: What will happen to ZFS Crypto? - Added by Jorgen Lundman over 2 years ago

I don't feel so confident that IllumOS doing own implementation is a good thing. As I assume, it will not be compatible with Solaris' crypto, we will then have 2 types of ZFS. Then, will the Linux ZFS kernel guys, or the MacOS ZFS build, take one of those branches, or also implement their own versions? One of the original goals with ZFS was architecture and endian independence, and it-is/has-been fantastic. I am definitely for maintaining the ability to use pools between platforms. I see I can even do read-only ZFS on Windows.

Meanwhile, it could be entirely possible someone not-related to IllumOS in any way, simply compile a package that can easily be installed after the fact. (kernel+userland binaries). It would get zfs-crypto usable by the consumers, and gives time to the gurus to either decide what to do, or implement their own version.

RE: What will happen to ZFS Crypto? - Added by Linda Kateley over 2 years ago

Jorgen,

When linux started getting popular in the mid 90's, solaris people were saying exactly what you are saying here... we used to ask "in a community, how do you ensure quality.. consistency?" What we found out is that all of the development adds to the core. I think we hope for forks.. we hope for additional communities. The zfs people are working on labeling the pools so when you import you can tell what features are available on the pool.. The core problem of mulitple distro's drives innovation and forces people to push beyond what is common.

Currently there are like 6-7 illumos based distros, that we know of.... I suspect there will be many more. How many linux distro's are there?

We hope illumos will be used. If you are creating an appliance, why wouldn't you use illumos? it has dtrace, fault management, zfs, zones.

RE: What will happen to ZFS Crypto? - Added by Charles Morris over 2 years ago

I completely sympathize with your concerns Jorgen. I would normally never encourage userland fork development that wasn't blessed by Sun, as in the case of OpenSolaris, but as we all know this is no longer a reality. Even community-sourced code is better than dealing with the alternative, meaning once-in-a-blue-moon and probably unintentional code drops, shoddy support, development without knowing the roadmap, legal issues, and being lied to and ignored.

I also don't see a reason why we couldn't make a best effort to provide a reverse-engineered Oralaris implementation, but that seems like a secondary objective. The primary objective is to
provide a solid ZFS implementation. That can't be done by depending on others that have zero desire to help you.

</opinionated unix user's opinions>

(1-19/19)