I used aws autoscaling for my webserver and Apache whirr for hadoop habse cluster. Do Apache whirr has an autoscaling property.
Is there any way to create an autoscaling Hadoop Hbase cluster
658 Views Asked by najeeb At
1
There are 1 best solutions below
Related Questions in HADOOP
- Can anyoone help me with this problem while trying to install hadoop on ubuntu?
- Hadoop No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster)
- Top-N using Python, MapReduce
- Spark Driver vs MapReduce Driver on YARN
- ERROR: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "maprfs"
- can't write pyspark dataframe to parquet file on windows
- How to optimize writing to a large table in Hive/HDFS using Spark
- Can't replicate block xxx because the block file doesn't exist, or is not accessible
- HDFS too many bad blocks due to "Operation category WRITE is not supported in state standby" - Understanding why datanode can't find Active NameNode
- distcp throws java.io.IOException when copying files
- Hadoop MapReduce WordPairsCount produces inconsistent results
- If my data is not partitioned can that be why I’m getting maxResultSize error for my PySpark job?
- resource manager and nodemanager connectivity issues
- ERROR flume.SinkRunner: Unable to deliver event
- converting varchar(7) to decimal (7,5) in hive
Related Questions in AMAZON-WEB-SERVICES
- S3 integration testing
- How to get content of BLOCK types LAYOUT_TITLE, LAYOUT_SECTION_HEADER and LAYOUT_xx in Textract
- Error **net::ERR_CONNECTION_RESET** error while uploading files to AWS S3 using multipart upload and Pre-Signed URL
- Failed to connect to your instance after deploying mern app on aws ec2 instance when i try to access frontend
- AWS - Tab Schema Conversion don't show up after creating a Migration Project
- Unable to run Bash Script using AWS Custom Lambda Runtime
- Using Amazon managed Prometheus to get EC2 metrics data in Grafana
- AWS Dns record A not navigate to elb
- Connection timed out error with smtp.gmail.com
- AWS Cognito Multi-tenant Integration | Ok to use Client’s Idp?
- Elasticbeanstalk FastAPI application is intermittently not responding to https requests
- Call an External API from AWS Lambda
- Why my mail service api spring isnt working?
- export 'AWSIoTProvider' (imported as 'AWSIoTProvider') was not found in '@aws-amplify/pubsub'
- How to take first x seconds of Audio from a wav file read from AWS S3 as binary stream using Python?
Related Questions in HBASE
- Apache atlas UI not showing up
- HBase Zookeeper Connection Error Docker Standalone 2.3.x and 2.4.x
- How does bulkload in databases such as hbase/cassandra/KV store work?
- How to eradicate the slowness caused due to reading rows from bigtable with hbase client in google dataflow job?
- i cant delete the specific column data by Timestamp
- hbase shell QualifierFilter is not filtering out columns when used with logical OR and SingleColumnValueFilter
- Spark - Fetch Hbase table all versions data using HBase Spark connector
- Unable to recover inconsistency in Hbase
- hBase java api, error on bulkload Added a key not lexically larger than previous sort (with JavaPairRDD<ImmutableBytesWritable, KeyValue>)
- Functionality inside completable future is completing quickly but completable future and timelimiter are taking too long
- about hbase put attribute
- java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/client/Table
- Big Table Java Connectivity issue
- How to check if the Thrift is working on HBase version 2.5 and How to indicate if Thrift 1 or Thrift 2 is installed?
- HMaster stuck at "Initialize ServerManager and schedule SCP for crash servers"
Related Questions in APACHE-WHIRR
- Booting a Druid Cluster using Whirr
- Cant start whirr on ec2 - describeAvailabilityZonesInRegion(Unknown Source)
- Yarn in Aazon EC2 with whirr
- apache hama on EC2 through whirr 0.8
- Whirr: hadoop-proxy.sh not working
- How do I make a patch from a github repo if I failed to make a feature branch?
- whirr unable to connect to custom AMI - EC2
- Is there any way to create an autoscaling Hadoop Hbase cluster
- AWS instance was deployed with Whirr. How do I get a key pair of the instance?
- Whirr could not download hadoop
- JPS not showing hbase or hadoop?
- Apache Whirr on EC2 with custom AMI
- How to install JDK 7 on EC2 cluster via whirr?
- Launch EC2 cluster with Whirr
- Error Installing Whirr using Maven
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
You can use Apache Mesos which a container based framework. Mesos works on container(e.g. dockers to start the services.) In mesos you can configure number of instances you need, configure resources, scale up/down etc. It will also take care of updating maintaining state of the process. This has been recently published hbaseconf 2015 where adobe has started using mesos for auto scaling in their production. You can also use google kuberneetes or marathon for managing the cluster and configurations. I have personally tried this on my local cluster. For more information to get started please follow this link
Basic introduction and features of the mesos for quick reference [source: apache site]
What is Mesos? A distributed systems kernel
Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elastic Search) with API’s for resource management and scheduling across entire datacenter and cloud environments.
Project Features