These open source storage apps can help businesses or home users set up a storage appliance, build their own cloud storage service, backup their systems and more.
Is there a need for open source storage applications? Clearly.
As the volume of digital data continues to grow, storage becomes a bigger and bigger problem for both home users and organizations. According to the latest Digital Universe study by IDC, 4.4 zettabytes (4.4 billion terrabytes) of digital information existed in 2013, and the amount of data is doubling every two years. That means by the end of 2015, there will be 8.8 zettabytes in our world, and by 2010, there will be 44 zettabytes, enough to fill a stack of tablets stretching to the moon and back six times.
To cope with all this storage, people need digital storage space – and plenty of it. More and more frequently, they are turning to open source applications to help them set up and manage their storage. Some are attracted by the low cost of do-it-yourself storage, while others like the security and privacy or other features offered by open source projects.
Recently, there's been something of a surge in open source cloud storage and software-defined storage projects, so we've updated our previous list of open source storage solutions with quite a few new additions. If you'd like to call our attention to any other open source storage projects we may have missed, feel free to make note in the comments section below.
NAS/SAN Software
1. Ceph
Ceph offers distributed block and object storage capabilities, as well as a POSIX-compliant network file system. It boasts "high performance, large data storage, and maximum compatibility with legacy applications." Operating System: Linux
2. FreeNAS
Used by the United Nations, the Salvation Army, Reuters, Michigan State University, the Department of Homeland Security, Disney Interactive Media and many other organizations, FreeNAS makes it easy to turn standard hardware into a network attached storage device. Businesses that need paid support can purchase TrueNAS, an appliance that uses the same technology. Operating System: FreeBSD
3. NAS4Free
Similar to FreeNAS, NAS4Free allows users to create their own NAS appliances using standard hardware and the free download. Features include a web-based interface, ZFS v5000 support, ZFS LZ4- compression, disk encryption and software RAID 0, 1, 5 and mix. Operating System: FreeBSD
4. Openfiler
Used by organizations like Motorola, Pratt & Whitney, BillMeLater and the London Metropolitan Police, Openfiler unites NAS features like NFS, CIFS and HTTP) with SAN features like iSCSI, FC. Commercial support is available. Operating System: Linux
5. OpenSMT
This is another option for building your own NAS/SAN appliance—this time based on OpenSolaris. It uses the ZFS file system. Operating System: OpenSolaris
6. Open Media Vault
Aimed at home users or SMBs, Open Media Vault allows users to build a Linux-based NAS appliance. It includes a built-in BitTorrent client and DAAP server, as well as support for SSH, (S)FTP, SMB/CIFS and Rsync. Operating System: Linux
7. Turnkey Linux
At the Turnkey site, you'll find more than 100 images for building all kinds of open source appliances, including the Turnkey File Server for building a simple network attached storage device. It includes compression support, a Web-based GUI and support for SMB, SFTP and Rsync file transfer. Operating System: Linux
Filesystems
8. Btrfs
Btrfs is a newer Linux filesystem being developed by Facebook, Fujitsu, Intel, the Linux Foundation, Novell, Oracle, Red Hat and some other organizations. It emphasizes fault tolerance and easy administration, and it supports files as large as 16 EiB. Operating System: Linux
9. Ext4
This is the latest version of one of the most popular filesystems for Linux. One of its key benefits is the ability to handle very large amounts of data— 16 TB maximum per file and 1 EB (exabyte, or 1 million terabytes) maximum per filesystem. Operating System: Linux
10 GlusterFS
Owned by RedHat, GlusterFS is a scale-out distributed file system designed to handle petabytes worth of data. Features include high availability, fast performance, global namespace, elastic hash algorithm and an elastic volume manager. Operating System: Linux
11. Lustre
Designed for "the world's largest and most complex computing environments," Lustre is a high-performance scale-out file system. It boasts that it can handle tens of thousands of nodes and petabytes of data with very fast throughput. Operating System: Linux
12. ZFS
Incorporated into many other open source projects, including many Linux distributions, ZFS is a popular filesystem for large data stores. Features include storage pooling, transactional copy-on-write semantics, checksums, snapshots, simplified administration and excellent scalability. Operating System: Solaris, OpenSolaris, Linux, OS X, FreeBSD
RAID
13. DRBD
Owned by LinBit, DRBD allows users to build a high availability storage cluster using replication that is very similar to RAID-1. It offers fully synchronous, memory synchronous or asynchronous operation. Operating System: Linux
14. Mdadm
Part of the Linux kernel, Mdadm enables the management of Linux-based RAID arrays. It includes monitoring and reporting capabilities, as well as the ability to move data between arrays. Operating System: Linux
15. Raider
Raider is a bash shell script for building software RAID 1, 4, 5, 6, or 10 arrays. It's optimized for many popular Linux distributions, including Ubuntu, Debian, Ubuntu, Debian, Arch, Mandriva, Mageia, openSuSE, Fedora, Centos, PCLinuxOS, Linux Mint, Scientific Linux, Gentoo, Slackware, Bodhi Linux and others. Operating System: Linux
16. RaidEye
This app works with the built-in RAID capabilities of OS X to monitor storage arrays. It sends email and sound alerts when it detects failures. Operating System: OS X
17. Salamander
Salamander aims to simplify the process of installing Linux on a RAID array. The name comes from the fact that Salamander arrays can recover after a hard-drive failure, just like living salamanders can regrow limbs after one has been cut off. Operating System: Linux
18. SnapRAID
Designed for home media centers and similar storage solutions, SnapRAID provides data duplication such that your array will be able to recover from the loss of up to six disks. Operating System: Windows, Linux, OS X
Backup and Synchronization
19. AMANDA
Short for "Advanced Maryland Automatic Network Disk Archiver," AMANDA backs up multiple systems on a network to tape, hard disk or optical media. Zmanda, which sponsors the project, offers commercial products and cloud-based storage that uses the same technology. Operating System: Windows, Linux, OS X.
20. Areca Backup
If you need to back up a standalone system, Areca offers versatility and simple setup. Features include compression, encryption, archive merge, filters and support for incremental, differential, full and delta backup. Operating System: Windows, Linux
21. BackupPC
This enterprise-grade solution uses a unique pooling and compression scheme to minimize storage space requirements and I/Os. For example, backing up 95 laptops for three weeks may require 1200 GB of space using other backup methods, but with BackupPC, it would require only 150 MB. Operating System: Windows, Linux
22. Bacula
Also designed for enterprises, Bacula offers network-based backup, recovery and data verification capabilities. Commercial support, training and services are available throughBacula Systems. Operating System: Windows, Linux, OS X
23. Back In Time
Similar to FlyBack (see below), Back in Time offers a very easy-to-configure snapshot backup solution. GUIs are available for both Gnome and KDE (4.1 or greater). Operating System: Linux
24. Bareos
Short for "Backup Archiving Recovery Open Sourced," Bareos is a fork of Bacula that offers features like LTO hardware encryption, efficient bandwidth usage and practical console commands. A commercially supported version of the same software is available through Bareos.com. Operating System: Windows, Linux, OS X
25. Box Backup
Box Backup describes itself as "an open source, completely automatic, online backup system." It creates backups continuously and can support RAID. Operating System: Windows, Linux
26. BURP
BURP, which stands for "BackUp And Restore Program," is a network backup tool based on librsync and VSS. It's designed to be easy to configure and to work well with disk storage. Operating System: Windows, Linux
27. Clonezilla
Intended as a replacement for commercial partitioning, disk imaging and cloning solutions, Clonezilla comes in two versions: Live for backing up single machines, and SE for cloning or restoring multiple systems on a network simultaneously. It boasts multicast restores as fast as 8 GB per minute. Operating System: Linux
28. Create Synchronicity
Extremely lightweight (only 220KB), Create Synchronicity is a fast and easy backup solution. It's available in multiple languages, supports scheduling and is fully portable. Operating System: Windows
29. DAR
DAR stands for "Disk Archive." It's a mature, 14-year-old command-line tool that backs up directory trees and files. An optional GUI is available from the DarGUI project. Operating System: Windows, Linux, OS X
30. DirSync Pro
Describing itself as "small but powerful," DirSync Pro offers file and folder synchronization and mirroring capabilities. It includes incremental backup, filtering and scheduling capabilities. Operating System: Windows
31. Duplicati
Duplicati is designed to backup your network to a cloud computing service like Amazon S3, Microsoft OneDrive, Google Cloud or Rackspace. It includes AES-256 encryption and a scheduler, as well as features like filters, deletion rules, transfer and bandwidth options. Operating System: Windows, Linux, OS X
32. FlyBack
Similar to Apple's TimeMachine, FlyBack provides incremental backup capabilities and allows users to recover their systems from any previous time. The interface is very easy to use, but little customization is available. Operating System: Linux
33. FOG
Another imaging and cloning solution, FOG brings together several other open source applications and provides an easy-to-use Web-based interface. It runs on a Linux server, but can also backup Windows systems running on the network. Operating System: Linux, Windows.
34. FreeFileSync
This synchronization tool is useful for backing up standalone Windows, Linux and OS X systems. Features include copying locked files, conflict detection, binary file comparison, comprehensive error reporting, support for long file paths, 64-bit support and more. Operating System: Linux, Windows, OS X
35. FullSync
Designed to help Web developers publish and update websites, FullSync can also synchronize files between a PC and a backup device. It supports mulitple transfer protocols and offers flexible rules for determining which files to include or exclude. Operating System: Linux, Windows, OS X
36. Grsync
This tool adds a graphical interface to the Rsync command line synchronization tool. It can synchronize music files between a PC and a mobile device, backup files to a network, replicate partitions, mirror files and more. Operating System: Linux, Windows, OS X
37. LuckyBackup
Also powered by Rsync, LuckyBackup is fast and easy-to-use and has won a couple of awards. It creates snapshots of your system and enables you to roll back to past dates. Operating System: Linux, Windows
38. Mondo Rescue
Packaged for many of the most popular Linux distributions, Mondo Rescue is a disaster recovery solution that supports tapes, disks or optical media. Its users include Lockheed-Martin, Nortel Networks, Siemens, HP, IBM, NASA's JPL and the U.S. Dept. of Agriculture. Operating System: Linux
39. Obnam
This app performs snapshot backups that can be stored on local disks or online storage services. Key features include de-duplication, encryption and push or pull encryption capabilities. Operating System: Linux
40. Partimage
Partimage is useful for installing multiple, identical images on several different PCs or for creating, copying or restoring partitions on a drive. It promises fast and easy restores. Operating System: Linux
41. Redo
Redo aims to be the "easiest, most complete disaster recovery solution available. It can do bare metal restores in as fast as ten minutes, and it also allows users to recover files or photos they have accidentally deleted. Operating System: Windows, Linux
42. Rsync
This widely used utility offers fast incremental file transfer, making it useful for backup situations. It can use rsh, ssh or direct sockets as the transport. Operating System: Linux, Windows, OS X
43. Synkron
Synkron is a simple cross-platform synchronization tool. It offers multiple configuration options and lets you blacklist files you don't want to be copied. Operating System: Windows, Linux, OS X
44. Unison
This file synchronization tool goes beyond the capabilities of most backup systems, because it can reconcile several slightly different copies of the same file stored in different places. It can work between any two (or more) computers connected to the Internet, even if they don't have the same operating system. Operating System: Windows, Unix
45. UrBackup
This file-server backup solution creates both image and file backups with an eye to data safety and fast restores. It's easy to setup, and it automatically backs up your system while it's in use. Operating System: Windows, Linux
46. Weex
Like some of the other tools on this list, Weex was created for Web developers who needed an easy way to update their website files on their servers. However, it can remotely update and synchronize any types of files. Operating System: Windows, Linux
Online/Cloud Data Storage
47. Camlistore
Camlistore, short for "Content-Addressable Multi-Layer Indexed Storage," describes itself as "a set of open source formats, protocols, and software for modeling, storing, searching, sharing and synchronizing data in the post-PC era." Note that the project is still under very active development, and the release still has quite a few bugs. Operating System: Linux
48. CloudStack
Apache's CloudStack project offers a complete cloud computing solution, including cloud storage. Key storage features include tiering, block storage volumes and support for most storage hardware. Operating System: Linux
49. CloudStore
Conceived as an alternative to Dropbox, CloudStore is a file synchronization app that focuses on speed and security. Setup takes just seconds but it is highly flexible. (Note that this is not the UK government initiative of the same name.) Operating System: Linux
50. Cozy
This personal cloud solution allows users to "host, hack and delete" their own files. It stores calendar and contact information in addition to documents, and it also has an app store with compatible applications. Operating System: Linux
51. FTPbox
With FTPbox, users can set up their own servers to act as a cloud storage solution. It offers the convenience of cloud storage without the risk inherent in trusting a third party with your data. Operating System: Windows
52. LipSync
Less polished than the other entries on the list, LipSync is a simple tool for synchronizing files stored on two or more hosts. To learn more about it, see the developer's blog post on How to Build Your Own Dropbox Clone. Operating System: Linux
53. OpenStack
OpenStack is a full-featured cloud computing platform, which includes object and block cloud storage capabilities. Several commercial providers offer cloud products based on OpenStack technology. Operating System: open source
54. PerlShare
Another Dropbox alternative, PerlShare allows users to set up their own cloud storage servers. Windows and OS X support is under development, but it works on Linux today. Operating System: Linux
55. Pydio
Pydio, previously known as AjaXplorer, offers enterprise-class file syncing and sharing. It aims to offer the same ease of use found in popular cloud storage services. Operating System: Windows, Linux (Android and iOS clients available)
56. SeaFile
SeaFile offers open source cloud storage and file synchronization. You can self-host with the free community or paid professional editions, or you can pay for the service hosting either in Germany or on Amazon's cloud in the U.S. Operating System: Windows, Linux, OS X, Android, iOS
57. SparkleShare
Another option for self-hosted storage, SparkleShare is great for keeping frequently changed files up to date. It offers built in encryption and the ability to revert to previously saved versions of a file. Operating System: Windows, Linux, OS X
58. Syncany
Still an alpha release, Syncany allows users to sync and share data using any kind of storage, including public cloud services. All files are encrypted before uploading in order to ensure privacy. Operating System: Windows, Linux, OS X
59. Syncthing
Designed to replace Dropbox and similar services, Syncthing allows users to set up their own cloud storage servers. Key features include encryption, authentication and a Web GUI. Operating System: Windows, Linux, OS X
Storage Management
60. Libvirt Storage Management
Sponsored by Red Hat, this storage virtualization API allows administrators to set up storage pools and volumes. It supports all the major hypervisors and a variety of storage pool types. Operating System: Linux
61. OHSM
Short for "Online Hierarchical Storage Manager," OHSM can automatically move data between high- and low-cost media depending on how often it has been accessed. It allows enterprises to set up and enforce policies for where storage should be allocated and when it should be relocated. Operating System: Linux
62. openAttic
OpenATTIC is a software-defined storage platform that offers support for native protocol standards, block-based storage management, snapshot backups, data mirroring and more. It prides itself on being a full-featured storage management system. Operating System: Linux
63. openvStorage
Similar to similar to VMware vSAN, openvStorage describes itself as "an open-source VM storage router for virtual machines." It helps organizations virtualize and manage their storage, while providing a great deal of flexibility and scalability. Operating System: Linux
64. oVirt
Powered by KVM, oVirt is a full virtualization platform that manages virtual machines and networks as well as virtualized storage. Features include high availability, live migration, load balancing, a Web-based interface and more. Operating System: Linux
Compression
65. 7-zip
In order to store archived files in as little space as possible, you need to compress them first. 7-zip offers a compression format 2-10 percent more efficient than other compression utilities like WinZip. Operating System: Windows, Linux, OS X
66. ArchivConvert
Winner of multiple awards, ArchivConvert does just what its name suggests—converting archive files from one format to another. It can read 24 different formats and write to 6. Operating System: Windows
67. ArcThemALL!
This compression utility can extract form 33 different formats and write to Zip, 7Z or UPX formats. It can also create self-extracting archives and includes encryption capabilities. Operating System: Windows
68. Keka
For Macs only, Keka can extract 12 different archive formats and create 7Z, Zip, Tar, Gzip, Bzip2, DMG or ISO files. In addition to the link above, it's also available through the Mac App Store. Operating System: OS X
69. PeaZip
PeaZip supports an astounding 150+ archive file formats. It also includes strong encryption, two-factor authentication and secure deletion capabilities for privacy protection. Operating System: Windows, Linux, OS X.
Data Destruction
70. BleachBit
If you want to be sure that no one can ever recover files you've deleted from your storage system, BleachBit offers secure deletion (shredding) capabilities. It can also protect privacy and help PCs run faster by cleaning out cache, deleting temporary logs, getting rid of cookies, erasing browsing history and more. Operating System: Windows, Linux
71. Darik's Boot And Nuke
To complete erase an entire drive, run DBAN. Commercial services and products, including erasure of RAID arrays, is available through Blancco, the project owner. Operating System: OS Independent
72. Eraser
Eraser securely deletes individual files. It overwrites data so it cannot be recovered, and it includes a customizable scheduler. Operating System: Windows
73. FileKiller
Similar to Eraser, FileKiller completely eliminates all traces of individual files that you want to delete. It allows users to select their level of privacy by determining for themselves how many times erased data should be overwritten. Operating System: Windows
Distributed Storage/Big Data Tools
74. BigData
Big data describes itself as "an ultra high-performance graph database supporting the RDF data model." It can scale to 50 billion edges on a single machine. Commercial support is available. Operating System: OS Independent
75. Hadoop
Apache's Hadoop project is so well known that it has become nearly synonymous with big data. It offers distributed storage and processing of very large data sets. Operating System: Windows, Linux, OS X
76. HPCC
Intended as an alternative to Hadoop, HPCC, or high performance computing cluster, is a distributed data storage and processing platform that scales to thousands of nodes. It was developed by LexisNexis Risk Solutions, which also offers paid enterprise versions of the software. Operating System: Linux
77. Sheepdog
Able to scale to thousands of nodes, Sheepdog offers distributed object storage for volume and container services. It's compatible with OpenStack Swift and Amazon S3. Operating System: Linux
Inside TEMPlate====>
Full Site
About
RSS