Enabling FUSE on your VPS (or how to use Amazon S3 on your VPS!)

O

opoloko

Member
Hi,

I'm trying to deal with this with support (350299) but it all seems shady and confusing and I cannot get a reply. CentOS has support for FUSE, that comes as a simple module and does not require any recompilation of kernel.

Using my VPS yum and the standard CentOS repository I installed all FUSE packages (bin, dev, libs) without any problem, yum took care of dependencies and all was ok. I also restarted the VPS.

Problem is that performing a modprobe -l it seems there are no modules installed, and I cannot load FUSE module. Reading on the internet I read that some hosting companies limit this by default and I only need to have it enabled. Is this the case? For me is fundamental as we share a company folder, and it always worked on another VPS I have with Arvixe. I use WHM/cPanel for some configurations but I always then manage system installations via yum or compiling sources.

Can someone help me and enable FUSE (and/or in general modules) on my VPS? You can see it's all installed correctly with "yum list installed".

Thanks!
 
Hi,

Since KH VPS's run on Virtuozzo, and Virtuozzo containers get their kernel from the host node, it is not possible to load kernel modules from a container. It is not supported by the software and we cannot make changes to the host node's kernel for specific VPS's for obvious reasons. If you need that level of control then you should look into dedicated servers. Also, if you simply need a way to share files then you can use webdav. This is included in cPanel as the feature "Web Disk".
 
Hi Mat,

thanks for explanation, I understand the problem. As I was explaining in the ticket when I worked with Virtuozzo FUSE was part of standard kernel, so once installed the modules on my container would be just a matter of one line command to enable it on my container.

I understand that would mean to remember this once you move thing around and could be complicated, but I was hoping in some kind of personalisation available consider is possible with Virtuozzo (I understand that if not already enabled on your kernel you'd need to recompile with that module on the node).

I know cPanel option but I don't need to access my server in webdav, but to access my webdav disk FROM my server, so I either need FUSE or Coda enabled. Honestly upgrading to a dedicated only for one single Kernel module is too much for the price, so at the moment I'm stuck about that.

Do you know if you have any SSD node that has already FUSE enabled? In that case could be easier.
 
Hi,

Can you put what your needs as far as file sharing are into context? I think if I understand what you are needing to do as far as file sharing, then I may be able to help you find a solution other than fuse if you are open to it. IE: Are you storing backups remotely, backing up a web site, publishing documents to web, etc.
 
Thanks Mat, that's very kind of you :)

Basically I had a webfuse connected cloud storage service for different reasons: 1) backing up some important or big documents 2) offer to a client of mine on their own domain a way to integrate a storing service without taking space on my disk 3) I host a few music bands/labels website and they offer sometimes limited downloads of high resolution files, and again they can take lots of space 4) I work in a recording studio as well and I need sometimes to let my client download their mix and I always do it with a folder on my own website that looks more professional and they feel protected

Bandwidth is not a problem because they are not downloads going on several times, but storage space was an issue on HG, where I had just 30gb, just like now on my SSD, and I need sometimes 20gb more all together or even more, so a cheap cloud service connected via webdav and mounted locally allows me to do it in a transparent way or to move files around (depending on the speed/space/download I need), extending my space without clogging my server space.

For a store I host that was having lots of downloads/bandwidth I integrated an Amazon S3 into their php code and it's ok, but I'd still need to have another 20gb spare to use when needed, and adding your SSD space it's just too expensive at the moment for me :) (don't get me wrong, loving your service and servers!!).

I don't know if this gives you some more information..basically a could drive connected via webdav on my server to expand my available space for big files/downloads/files movement.
 
Hi Mat,

let me know if you have any idea/suggestion about this, I'm holding the ticket open (but from there I just got the answer that is not possible for me to load the module kernel, no alternatives/alternative ideas.

Thanks!

p.s. and today another ticket opened, solved and verified (including explanation for the future!) all within an hour!
 
Hi,

I see what you mean. I was hoping that you were just syncing backups or something like that to another server. I don't think you will find anything suitable for doing that sort of mount that does not require fuse I know that sshfs and ftpfs both do. I know that you don't want a dedicated server for this one site, but if you did ever go to a dedicated you could eliminate the need for the cloud storage and even use it as remote storage for other servers that you have and/or sell backup space on it. It may be something to keep in mind for the future.

You may also want to look into using a CDN to host the content. I don't know how your clients are connecting and accessing the data. If they are using webdav to your server themselves then it wouldn't help, but if the content is being served via a php script or static html, then you could possibly host the content at a CDN and link to it in your app.
 
Hi opoloko,

I don't know if it's an alternative or not but I know that you can use Amazon's S3 to store files and with DNS have it look as though the files are still coming from your domain. Perhaps you could look into that.
 
Hi Mat,

I considered sshfs and ftpfs but they are all based on FUSE anyway, I really hope in the future there will be support for it as it allows lots of things. I'll take into consideration dedicated, but for now it's not an option :) The idea of the CDN was already there but most of all to speed up some static content, not really for the space issue, but thanks again for ideas and positive attitude.

Dan actually considering that I already used with some scripts S3, I may think just "mapping" my existing "exchange" folder with a rewrite of Apache and transfer all existing material on S3 so I can free space and server it from there giving out anyway links to my website and having the files coming from a subdomain of mine.

That's a good idea, FUSE would me amazing to have in the future (I understand why it's not at the moment) but in the meantime this solution will work probably, anyway better than now!
 
Done, 2hr and already working :)

Hopefully it will be useful for other KH customers looking for expanding their hard drive space at a very cheap price and at the same time saving on bandwidth and space, all within their domains and doing it all from the VPS shell. It's good to expand your space or to move some stuff from your VPS to Amazon S3 for storage and serving from your website, or even to backup stuff. My personal thank you to everyone here :)

I have no time of writing a detailed guide, but I'll give all steps:

1) open an account on Amazon S3 and understand how it works and how to operate it's console (for steps 2-3 of this guide)...it's easy! You should get some free stuff as you start and then it's ridiculously cheap and fast..use their calculator to get an idea. Basically you open an account, you can create in this several 'buckets' (imagine them as containers), and in each of them put as many folders/files you want (single file size size up to...5 terabytes..should be more than enough)
2) create a bucket named like yoursubdomain.yourdomain.com
3) allow static hosting on it, and make its policy to turn all uploaded files public (follow this guide http://www.emind.co/make-s3-bucket-public/). Otherwise you have to Make Public every file you upload. When you make a bucket for static hosting, you'll get an endpoint..write it somewhere!
4) create a CNAME record in the DNS zone of your domain: name "yoursubdomain", host "yoursubdomain.yourdomain.com.s3.amazonaws.com." (<- yes the final dot you need it!)
5) consider the time for DNS to propagate..if the subdomain is new it could be very fast just a few minutes if you live in a big country (from here UK it took 1 minute)
6) install on your VPS the command line s3cmd (http://s3tools.org/download): with that you have a command line tool to list, move, read, write, sync etc.
7) as a test put a file in your S3 bucket, and try accessing it first using http://endpoint/<nameoffile> (see point n.3 above), and verify it works: if not check bucket policy or file not being public..back to point 3 above! :) If it works, try again but with http://yoursubdomain.yourdomain.com/<nameofthefile>, and you'll download from your domain name...but from Amazon S3.
8) if like me you already had a folder for example with lots of files for your visitors, let's say a folder called "files" (so people were downloading from yourdomain.com/files/<nameoffile>), you can keep all the links untouched and automagically and transparently redirect calls on your Amazon S3. Open your .htaccess for yourdomain.com, and write a single rule:
RewriteRule ^files/(.*) http://yoursubdomain.yourdomain.com/$1 [QSA,R=301]
9) now try calling yourdomain.com/file/<nameoffile> and without changing a link, again you're downloading from your domain but in reality from Amazon S3

You can have as many buckets as you want in your S3, and you can repeat/vary this procedure for all the folders and subdomains/domains you want. This way you can move lots of stuff on S3 and using it as storage and file server, all masked anyway under your domain(s).

Obviously you can create a bucket without making it public, and use it as a place where you backup for yourself all your VPS or even more (remember the command s3cmd...you can script that). Pushing it further, you can easily mount your Amazon S3 as an hard drive on your desktop, and use it as well as backup and to access/exchange files.

Sorry for instructions not really detailed, if you have the knowledge you'll be able to follow them and do it, if you're lost maybe you should ask help of someone to help you...for me it took 2hrs to do everything including moving files, it's already working and I freed up several gigabytes on my precious SSD VPS!
 
Awesome! Thanks for the writeup opoloko!

They have really simplified this in the past 2 years, glad to see it went so well for you :D
 
You must log in or register to reply here.
Top