Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Cloud Data Storage Facebook Linux IT

Facebook To Begin Deploying Btrfs 115

Posted by timothy from the so-rich-and-smooth dept.
An anonymous reader writes "After hiring the lead Btrfs developers and Linux kernel block maintainers last year, Facebook is beginning trial deployments of Btrfs. Facebook will start using the next-generation file-system within their web-tier and they will be among the first major public deployments of Btrfs."
This discussion has been archived. No new comments can be posted.

Facebook To Begin Deploying Btrfs More

Facebook To Begin Deploying Btrfs

Comments Filter:

  • Give me a big plate of beans (Score:4, Funny)

    by Anonymous Coward on Thursday March 27, 2014 @02:05PM (#46594553)

    and I'll start deploying all kinds of btrfs.

  • How about real problems (Score:5, Funny)

    by i kan reed ( 749298 ) on Thursday March 27, 2014 @02:07PM (#46594573) Homepage Journal

    When are they going to make the users of their website tolerable human beings instead of insane caricatures designed to make you lose all faith in humanity?

    • Re: (Score:3, Insightful)

      by Anonymous Coward

      Bad news, those are the real human beings. The tolerable ones you want to rule the place are actually marketing sockpuppets.

    • Unfortunately, I believe that is the role of the user. I don't think that Facebook would try re-routing users to code.org [code.org].

    • Re: (Score:1)

      by Anonymous Coward

      Get better friends.

    • Re:How about real problems (Score:5, Funny)

      by gstoddart ( 321705 ) on Thursday March 27, 2014 @02:45PM (#46594891) Homepage

      When are they going to make the users of their website tolerable human beings instead of insane caricatures designed to make you lose all faith in humanity?

      And ... this is different from Slashdot, how? ;-)

    • Re:I guess Minecraft will stop using it (Score:5, Interesting)

      by vux984 ( 928602 ) on Thursday March 27, 2014 @02:17PM (#46594687)

      If Facebook likes something, it must be evil.

      If facebook buys something outright, it will be turned to social-shit. FTFY.

      If I happen to share Zuckershits taste in coffee makers, that won't affect my enjoyment of the coffee maker. On the other hand, if Facebook buys out my favorite coffee maker... then yeah, my next coffee maker will be a brand that doesn't try and change my status to "making facebook coffee" everytime I brew a cup. (Well... I don't have a facebook account so that's moot... although I expect a facebook coffeemaker would require one before allowing coffee to be brewed so... there's that.)

      • Well they "bought out" (that is, "hired") the lead Btrfs developers, so how do you count that?
      • Why do you think that? Both Instagram and Whats App are working as they originally did prior to their acquisition.
        • For now. How long did it take Google to begin forcing G+ integration onto Youtube?

          • Well, considering G+ didn't exist when Youtube was purchased... Its not that good of an analogy.

            Try comparing it to the Microsoft skype purchase.

            Funny note, Microsoft is now hosting developer video hangouts over G+. Despite owning skype outright, and owning a part of facebook.

            I also hate the Youtube G+ Integration, for the opposite reason. G+ is great, youtube and the comments there are horrible and shouldn't be brought to light anywhere civilized people gather.

        • Re: (Score:3)

          by vux984 ( 928602 )

          Both Instagram and Whats App are working as they originally did prior to their acquisition.

          Well except where everything you do is fed into facebook's data mining backend, and its only a matter of time before they add a back channel to start showing you facebook ads.

          They will monetize it. Its just a matter of time.

          • Re: (Score:2)

            by rev0lt ( 1950662 )
            So, you're saying that would never happen if they stayed independent? Such innocence...

            • Re: (Score:2)

              by vux984 ( 928602 )

              So, you're saying that would never happen if they stayed independent?

              It would be instagram ads and whatsapp ads. Not facebook ads. That's a big difference.

              The amount of information whatsapp has on me is pretty limited compared to the profiles facebook or google are trying to build. I'm ok with small individual companies having small limited profiles of my activity with that company. I strongly object to the macro-scale surveillance that google and facebook attempt.

              • Re: (Score:2)

                by rev0lt ( 1950662 )
                Well, I'm no expert, so take what I'm about to say with a grain of salt, but Whatsapp would never be profitable as a standalone company with their current business model. And what you call pretty limited, its not when cross-referenced with eg. your cellphone number (Facebook, Google and VK all "encourage" you to give this). For many people, a cellphone number is way more stable than a facebook profile. And whatsapp (either the app or the server) keeps full blown logs of all conversations, so its quite easy

                • Re: (Score:2)

                  by vux984 ( 928602 )

                  Meh, I'd still rather it be whatsapp than facebook. I perceived them to be much smaller, even if still fairly big.

                  But for what its worth I've never had a whatsapp account nor an instagram one, nor a facebook one. I find nearly all internet "social" to be a monumental waste of time bundled with massive privacy invasion and avoid it pretty thoroughly.

                  I do have a skype account though, and would love to find an alternative to that too. Because the ads bug me, and the attachment now to microsoft bugs me.

                  • Re: (Score:2)

                    by rev0lt ( 1950662 )

                    Meh, I'd still rather it be whatsapp than facebook. I perceived them to be much smaller, even if still fairly big.

                    I understand that feeling. But its not based on facts or even objective... People often perceive big companies as "evil", and forget that, in that regard, all companies are evil in some way or another.

                    But for what its worth I've never had a whatsapp account nor an instagram one, nor a facebook one. I find nearly all internet "social" to be a monumental waste of time bundled with massive privacy invasion and avoid it pretty thoroughly.

                    Well, I had a whatsapp account and a facebook account. While I tend to agree regarding the waste of time, it is also a way for me to connect with friends and colleagues from past companies, and often discuss technical stuff and what's happening in the world. Isolation is good often, but its not good always. An

                    • Re: (Score:2)

                      by vux984 ( 928602 )

                      all companies are evil in some way or another.

                      Yes, all companies are evil. Bigger companies have the capacity to do more evil.

                      And while I have no objection to the usual "build a real product, sell that product to customers, make money" business model, even if it leads to evil, Facebook (and social in general) have a business model i abhor. So there's that too.

                      And to top if off, I dislike Zuckerberg intensely as a person, have no respect for him, and wish to do nothing to enrich him, while there are any numb

                    • Re: (Score:2)

                      by rev0lt ( 1950662 )

                      And to top if off, I dislike Zuckerberg intensely as a person, have no respect for him,

                      So. these sound like personal feelings and not a rational evaluation; I',m fine with that, but its not really a valid argument. And regarding Facebok, I'd be really surprised if Zuckerberg had any kind of real control over the company.

                      here is the service, run it on your own servers, and we'll sell you support.

                      You realize that, without a centralized system, the social model would not exist, right?

                      Not the brand, but what the tool is and does. Is it a secure two way messenger that I have complete control over?

                      Its not. It's more of a 2-way p2p messaging system. And I'd take skype proprietary encryption over any SIP-based hardware device. I'd really doubt that what you describe even exists. And ev

                    • Re: (Score:2)

                      by vux984 ( 928602 )

                      So. these sound like personal feelings and not a rational evaluation

                      I don't know how you can presume to weigh in whether or not whether I like someone is "rational" or not.

                      You realize that, without a centralized system, the social model would not exist, right?

                      Good riddance. I can be social without "the social model". Because I was participating on forums before facebook. And IRC. And usenet. And email.

                      And I'd take skype proprietary encryption over [...]

                      Microsoft can and does monitor the chat. They collect

    • The Timeline:

      • The "cool" kids start using Btrfs.
      • Facebook starts using Btrfs.
      • Parents start using Btrfs.
      • Btrfs Jumps the Shark [wikipedia.org].
      • Everyone stops using Btrfs.

  • Sounds about right (Score:5, Funny)

    by Impy the Impiuos Imp ( 442658 ) on Thursday March 27, 2014 @02:13PM (#46594643) Journal

    > Btrfs

    tl;dr, I assume this is a button to let you tag people as a butterface?

  • Thanks, Facebook! (Score:4, Insightful)

    by bill_mcgonigle ( 4333 ) * on Thursday March 27, 2014 @02:16PM (#46594667) Homepage Journal

    FB admins - thank you for paying the developers for the open source work they do. I've been using flashcache with great success in one deployment for almost two years now and am looking to start with hhvm. I didn't even know about the block work.

    Obviously kudos to the developers too for spending valuable years on it as well.

    • http://hhvm.com/ [hhvm.com] from TFSite "an open-source virtual machine designed for executing programs written in Hack and PHP. HHVM uses a just-in-time (JIT) compilation approach to achieve superior performance while maintaining the development flexibility that PHP provides."

  • Trial by fire... (Score:4, Insightful)

    by mlts ( 1038732 ) on Thursday March 27, 2014 @02:19PM (#46594691)

    IMHO, this is a very good thing. btrfs doesn't have as many capabilities that ZFS or Storage Spaces/ReFS possesses.

    However, it is finally time that Linux has a filesystem that supports the latest/greatest enterprise features (deduplication and the ability to combat bit rot.)

    Realistically, it would be nice to see the native (not FUSE based) code from OpenZFS be included as an alternative, but the CDDL/GPL conflicts likely will make this a no-go.

    • ...and the ship's owner is named Larry Ellison. How do you expect him to be able to afford enough Ole Doc Washington's Patented Yacht Oil to win the America's Cup if he gives nice things away for free??

    • Re:Trial by fire... (Score:5, Informative)

      by mcrbids ( 148650 ) on Thursday March 27, 2014 @02:34PM (#46594797) Journal

      Realistically, it would be nice to see the native (not FUSE based) code from OpenZFS be included as an alternative, but the CDDL/GPL conflicts likely will make this a no-go.

      Well, isn't this your lucky day, then? ZFS on Linux works now, today, without the use of FUSE. [zfsonlinux.org] Nothing about the license conflicts prohibits use or distribution, just distribution together. I have ZFS/Linux servers in production right now, and they are quite stable. Starting with a vanilla install of CentOS, the instructions are roughly:

      1) Install the yum repo file.
      2) yum Install kernel-devel zfs
      3) Start the ZFS service.
      4) Start creating ZFS volumes....

      A reboot isn't typically necessary... (though not a bad idea)

      • Re: (Score:1)

        by Anonymous Coward

        Running zfsonlinux here for everything except / and /boot. The last officially supported Linux version is 3.11. I am guessing (hoping) that 0.6.3, the next version of zfsonlinux, will be released soon after Linux 3.14 (with support commensurate support). If you're OK waiting a few kernel versions, or you're OK running HEAD, you'll be happy with zfsonlinux. It has been very stable for me, though I throw a ton of ECC ram at it (like you're supposed to).

      • Re: (Score:2)

        by mlts ( 1038732 )

        I appreciate that link. That is very useful. It would be nice if it was part of EPEL directly. I thought the ZFS development was dead on this, but apparently there was a release last August.

        • It also has an active development community; the git repo has regular and frequent commits (for a filesystem). ZFS on Linux seems to test more and release less often -- a fact I appreciate as I haven't lost a single bit of data on my ZFS filesystems, but have lost entire btrfs filesystems multiple times. (Yeah, sure, btrfs is "experimental" and will eat your data... so why is Facebook even thinking about using it?)

          • Re: (Score:2)

            by mlts ( 1038732 )

            btrfs has been "experimental" for quite a long time (2009?) To compare, Linux went from MINIX's filesystem to ext2 in two years, which lasted quite a while. It takes time to get a filesystem going, but five years is almost an eternity in the computer world, and realistically, Linux should have chucked the LVM2/ext4 combination long ago for ZFS or something ZFS-like.

            The good thing is that with FB's devs hammering on btrfs, that will do nothing but improve things and get btrfs ready for prime time use in th

        • Re: (Score:3, Informative)

          by SirMasterboy ( 872152 )

          Check out: http://open-zfs.org/wiki/Main_... [open-zfs.org]
          http://dtrace.org/blogs/ahl/20... [dtrace.org]

          ZoL is very active and very up-to-date. All the versions and compatibility is in sync with Illumos (the main source of OpenZFS) and FreeBSD. You can create and move zpools between these 3 platforms seamlessly.

          2 of the main founders and creators of ZFS itself (who used to work for Oracle and wrote ZFS) who now work for Delphix and continue to improve OpenZFS (started with the last open release of Oracle ZFS) in Illumos and have ac

      • A reboot isn't typically necessary... (though not a bad idea)

        What is this? Windows?

      • How has performance been? I ran zfsonlinux for a few months 2-3 years ago, and on a single redundancy zvol with compression on I could never get sustained data transfer rates over the network (using samba) above about 35 MB/s. I switched to ZFS on FreeBSD and it easily hits 60-100 MB/s. (This is with deduplication off. While dedup is a great idea in theory, I have never seen such a performance hog. It dropped my transfer rates to about 15, 8, and 2 MB/s for best, average, and worst case. And yes befor

        • Re: (Score:2)

          by rev0lt ( 1950662 )
          A nice alternative to ZFS Dedup (or so I heard) is DragonFly's HammerFS. Matt uses a compromising approach that is less taxing on resources.

    • Not that anybody'll really notice, but I have a feeling that Facebook's backup and recovery system is queuing up for a stress test.

      Having lost data with BTRFS multiple times on my disk array (as recently as last month), I have no confidence in it. The best thing I can say about btrfs is is that it was able to tell me that it had lost data. Not many filesystems do that; but ZFS on Linux [zfsonlinux.org] has been rock solid for years, and not only tells me if data has been lost, but actually preserves the data as well.

      • Re:And facebook will be burnt (Score:4, Interesting)

        by mlts ( 1038732 ) on Thursday March 27, 2014 @03:16PM (#46595127)

        You are not the only person who have reported data loss on btrfs. Normally I wouldn't worry about Linux filesystems (even ext4 became rock solid after a while.) However, I worry about what I hear from people who use btrfs.

        One concern is that a filesystem can't check for bit rot by itself. True bit rot checking requires at least some working with the LVM layer to check CRCs, find a damaged sector and fix it. I've read that btrfs can catch some bitrot issues, (and please correct me if wrong), but it can't catch/correct anywhere near as much as ZFS or Storage Spaces + ReFS can. btrfs also uses a 32 bit CRC, rather than a 64 bit one.

        I'm hoping that Facebook's coders can find the issues with btrfs and squash them. There are not many companies with the sheer server use of FB, and if they can get it working solidly, btrfs should be more than ready for prime time for everyone else.

        • Re: (Score:2)

          by mcrbids ( 148650 )

          If BTRFS bit rot detection is anything like ZFS' then it most certainly can check for bit rot and correct it automatically.

    • If FB needed ZFS, wouldn't they have gone for FreeBSD? The cleanest up to date OS that supports this file system? If they're going w/ BTRFS, they might as well go w/ Linux.

    • Re: (Score:3)

      by s.petry ( 762400 )
      Well, since ZFS is available for Linux I had to wonder why there would be people making a fuss about btrfs. You bring up licensing which is an issue, and I'm guessing Oracle did not help the license issues, or possibly made the license issues worse.

      • I would think licensing wouldn't be much of an issue. Facebook probably maintain their own internal custom linux distro. GPL incompatibility between ZFS and the kernel presumably wouldn't be a problem as they wouldn't be distributing it to anyone else.

        I could be wrong though :)

        • Re: (Score:2)

          by s.petry ( 762400 )
          I happen to use ZFS and have really enjoyed it since it came stock in Solaris 10. I really never paid attention to btrfs, no need to do so.

      • Re: (Score:2)

        by Rich0 ( 548339 )

        Well, since ZFS is available for Linux I had to wonder why there would be people making a fuss about btrfs. You bring up licensing which is an issue, and I'm guessing Oracle did not help the license issues, or possibly made the license issues worse.

        Well, for starters btrfs plans to have features I need, like reshaping RAID, while ZFS has no plans (that I'm aware of) to add this feature. No, I'm not talking about adding/removing raid from a zpool - I'm talking about adding/removing drives from a RAID while maintaining redundancy while the filesystem is online. mdadm supports this, and so does btrfs (though doing anything with raid5/6 on btrfs is risky right now).

        The main strength of ZFS is its maturity/stability. Feature-wise, I'm sure it does somet

    • Re: (Score:2)

      by Rich0 ( 548339 )

      IMHO, this is a very good thing. btrfs doesn't have as many capabilities that ZFS or Storage Spaces/ReFS possesses.

      Beyond maturity, what is actually missing? When I look at the feature lists for both if anything it seems like btrfs has more features, like being able to reshape a raid. The last time I checked ZFS supported adding or removing a raid from a zpool, but not adding or removing individual drives from a raid (without degrading it). That is, you can't turn a 4-drive raid5 into a 5-drive raid5 without adding 5 drives and then removing 4.

      I'm certainly willing to believe btrfs is missing something, but it has a

      • Re: (Score:1)

        by Anonymous Coward

        RAID 5/6 is exactly what is needed. The industry seems to swing from advanced hardware RAID to having the OS handle this task. There was a time back in the Ultra 450 days when one had to have Veritas's LVM software because Sun machines would ship as "dumb" JBOD without a hardware controller... then the world moved to SANs, from there to cloud storage, and from there (now that SAN prices are through the stratosphere) back to having the OS do the striping/RAID.

        It is good for an OS/filesystem to have this ca

        • Re: (Score:2)

          by Rich0 ( 548339 )

          RAID 5/6 is exactly what is needed.

          Btrfs supports RAID 5/6, with reshaping (ZFS does not support the latter).

          Then there is the issue of bit rot. A ZFS scrub isn't just an online fsck... it goes through every single sector looking for corruption and either finding it... or if there is redundancy left, fixing it.

          Fully supported on btrfs. I do it weekly. All reads are of course checked, but a scrub checks all the disks asynchronously.

          Finally there is the issue of snapshots. With ZFS, I can mount a drive, snapshot the entire system, copy that snap onto the mounted drive, dismount it and be on my way, a backup done.

          Snapshots are fully supported on btrfs. You can also use send/receive with them which would be more efficient in this use case than just copying the snapshot (which copies all data and not just changes since the last snapshot).

          My question was what features does ZFS have which btrfs doesn't have? All of these a

    • Re: (Score:2)

      by sjames ( 1099 )

      I have been evaluating both recently. In the end, btrfs is mostly to be more capable. I say to be because at the moment, it is not yet mature and some of those capabilities are either absent or not working properly, especially RAID >0. ZFS seems much more mature and it's capabilities work now.

      Neither lost data in my brief tests where I abused them with hard resets and disappearing drives (in a VM). However, btrfs got to a point where anything touching it got stuck in the D state, so it might as well have

  • As far as I know, the web tier is basically read-only images of the services to be run. The updates and data are on the back end.

    So what, precisely, does using Btrfs in such a deployment prove? It's the stability of modified disks that is in question.

    • Re: (Score:1)

      by Anonymous Coward

      As far as I know, the web tier is basically read-only images of the services to be run.

      The web tier will serve cached copies of user pages generated as requests arrive, and then distribute these generated pages as load increases. Given that Facebook has about 18.5% of the species actively using the service every month you can assume there are tens of thousands of page gens per minute into the web tier caches. The storage elements used to achieve that will experience very frequent writes.

  • If they strap on an oculus rift headset, do you think that maybe they'd be able to see that nobody gives a shit unless it's news about them going bankrupt? I bet 50% of slashdotters don't use Facebook. They could use hamsters on wheels and mathematical savant unicorns as servers for all I care. I just want them to die already.

  • Btrfs definition (Score:5, Informative)

    by PHPNerd ( 1039992 ) on Thursday March 27, 2014 @03:00PM (#46595005) Homepage
    From wikipedia: Btrfs (B-tree file system) is a GPL-licensed experimental copy-on-write file system for Linux. (I'm sure a lot of people were wondering what it is, since TFA doesn't say)

    • Re: (Score:3)

      by dargaud ( 518470 )
      I hope the /. wannabes that'll spring up when beta goes into production will enforce the explicit declaration of all acronyms in the summary...
  • Thank you, ladies and gentlemen.

Slashdot Top Deals

BLISS is ignorance.

Close