Tag Archive for amazon

Change on Membase console the internal ip to external ip

As per the documentation on - Using+Membase+in+the+Cloud, we have followed the below steps to change from internal ip to external ip.

  1. Install the Membase software (currently this is only supported for Linux installations)
  2. Shut down the service
  3. Edit the script located at /opt/membase/1.6.0/bin/membase:
    1. Add a ‘\’ to the end of the last line
    2. Add a new last line that reads: «-name ns_1@<hostname>»
  4. Delete the files under:
    1. /var/opt/membase/1.6.0/data/ns_1/
    2. /var/opt/membase/1.6.0/membase/
    3. /etc/opt/membase/1.6.0/ns_1/
  5. Restart the service
  6. See the node correctly identifying itself as the hostname in the GUI under the Manage Servers page.


But it doesn’t seem to work for us. Infact we cannot see the management console on: http://xxxxxxxxx:8091/


Using Membase in the Cloud

For the purposes of this discussion, we will refer to «the cloud» as Amazon’s EC2 environment since that is by far the most common cloud-based environment.  However, the same considerations apply to any environment that acts like EC2 (an organization’s private cloud for example).  In terms of the software itself, we have done extensive testing within EC2 (and some of our largest customers have already deployed Membase there for production use).  Because of this, we have encountered and resolved a variety of bugs only exposed by the sometimes unpredictable characteristics of this environment.

Being simply a software package, Membase Server is extremely easy to deploy in the cloud.  From the software’s perspective, there is really no difference being installed on bare-metal or virtualized operating systems.  On the other hand, the management and deployment characteristics of the cloud warrant a separate discussion on the best ways to use Membase.

We have written a number of RightScale templates to aid in your deployment within Amazon.  You can very easily sign up for a free RightScale account to try it out.  The templates handle almost all of the special configuration needed to make your experience within EC2 successful.  Direct integration with RightScale also allows us to do some pretty cool things around auto-scaling and pre-packaged deployment.  Check out the templates here.

We’ve also authored an AMI for use within EC2 independent of RightScale.  When using these, you will have to handle the specific complexities yourself.  You can find this AMI by searching for ‘membase’ in Amazon’s EC2 portal.

Some considerations to take into account when deploying within the cloud are:

  • Local storage being ephemeral (requiring special attention for backups)
  • IP addresses of a server changing from runtime to runtime
  • Security groups/firewall settings
  • swap

Local Storage

Dealing with the first point is not very much different than a data center deployment.  However, EC2 provides an interesting solution.  Through the use of EBS storage, you can obviate the largest concern of losing your data when an instance fails.  Writing Membase data and configuration to EBS creates a reliable medium of storage.  There is direct support for using EBS within RightScale and of course you can set it up manually yourself.

Using EBS is definitely not required, but you should make sure to follow the best practices around performing backups.

Keep in mind that you will have to update the per-node disk path when configuring Membase to point to wherever you have mounted an external volume.

IP addresses

The second issue is a bit trickier and requires configuring Membase to use a DNS entry instead of an IP address.  By default, Membase servers use their IP address as a unique identifier.  If the IP changes, an individual node will not be able to identify its own configuration and other nodes that it was clustered to will not be able to access it.  In order for a node to identify itself via a DNS name rather than and IP address, the following instructions must be followed.  Note that this configuration is automatically handled by the RightScale server template.

A few points to keep in mind when setting this up:

  • Make sure that this hostname always resolves to the IP address of the node that it is on.  This can be accomplished by using a dynamic DNS service such as DNSMadeEasy which will allow you to automatically update the hostname when an underlying IP address changes.
  • It is best to make sure that the IP address registered with the hostname is the internal address for the node (rather than the external one provided by Amazon) so that other nodes and application machines can contact it

The below steps will completely wipe any data and configuration from the node, so it is best to start with a fresh Membase install.  If you already have a running cluster, you can easily rebalance a node out of the cluster, make the change and then rebalance it back into the cluster.  Nodes with IP’s and hostnames can exist in the same cluster.

For Linux:

  1. Install the Membase software
  2. Execute: ‘sudo /etc/init.d/membase-server stop’
  3. Edit the script located at /opt/membase/1.6.0/bin/membase:
    1. Add a ‘\’ to the end of the last line
    2. Add a new last line that reads: «-name ns_1@<hostname>» (without the double quotes) where <hostname> is either a DNS name or an IP address that you want this server to listen on (the ‘ns_1@’ prefix is mandatory):
      ...rest of file...
      -detached \
      -run ns_bootstrap -- \
      -ns_server ns_server_config \"/etc/opt/membase/1.6.2/config\" \
      -name ns_1@membase1.company.com
  1. Delete the files under:
    1. /var/opt/membase/<version>/data/ns_1/
    2. /var/opt/membase/<version>/mnesia/
    3. /etc/opt/membase/<version>/ns_1/
  2. Execute: ‘sudo /etc/init.d/membase-server start’
  3. See the node correctly identifying itself as the hostname in the GUI under the Manage Servers page (you will be taken back to the setup wizard since the configuration was cleared out, but after completing the wizard the node will be named properly).

For Windows:

  1. Install the Membase software
  2. Stop the service by running C:\Program Files\Membase\Server\bin\service_stop.bat
  3. Unregister the service by running C:\Program Files\Membase\Server\bin\service_unregister.bat
  4. Edit the script located at C:\Program Files\Membase\Server\bin\service_register.bat:
    1. On the 7th line it says: set NS_NAME=ns_1@%IP_ADDR%
    2. Replace «%IP_ADDR%» with the hostname/IP address that you want to use
  5. Register the service by running the modified script: C:\Program Files\Membase\Server\bin\service_register.bat
  6. Start the service by running C:\Program Files\Membase\Server\bin\service_start.bat
  7. See the node correctly identifying itself as the hostname in the GUI under the Manage Servers page (you will be taken back to the setup wizard since the configuration was cleared out, but after completing the wizard the node will be named properly).

Security groups/firewall settings

It’s important to make sure you have both allowed AND restricted access to the appropriate ports in a Membase deployment. Nodes must be able to talk to one another on various [ports] and it is important to restrict external and/or internal access to only those individuals you want to have access. Unlike a typical data center deployment, cloud systems are open to the world by default and steps must be taken to restrict access.


Certain cloud systems by default don’t have a swap partition configured. While a system should not utilize a swap partition heavily, it is our recommended practice to have at least some amount of swap configured to avoid the kernel from killing processes.


Installing FUSE + s3fs and sshfs on Ubuntu

через s3fs не видно папки созданные через вэб интерфейс S3 или при помощи других программ!
папки созданные при помощи s3fs видно при помощи других программ.

Вот не плохая программа для Windows http://s3browser.com/

Installing FUSE + s3fs and sshfs on Ubuntu

Fuse is a program to mount «foreign» filesystems to your computer. This is how I got an Amazon S3 (simple storage service) bucket mounted on my Ubuntu server as if it were a local folder.

Install the necessary libraries:

sudo aptitude install build-essential libcurl4-openssl-dev libxml2-dev libfuse-dev comerr-dev libfuse2 libidn11-dev libkadm55 libkrb5-dev libldap2-dev libselinux1-dev libsepol1-dev pkg-config fuse-utils sshfs

We also installed the sshfs (SSH File System) in order to test out FUSE. Go ahead and create a mount folder and mount a remote drive to it:

sudo mkdir -p /mnt/sshfs

sudo chown user:fuse /mnt/sshfs

sudo sshfs user@hostname:/path/on/remote/server /mnt/sshfs

You should now be able to browse files on /path/on/remote/server, locally from /mnt/sshfs. Please note that you will need to set the ownership of the mount before you’ll be able to use it. The fuse group was created automatically and is a good place to start.
Read more