Re: Unable to create a table in hive
@Sivakumar Mahalingam,I guess the problem is with quotes. use ' instead of ’
View ArticleRe: pyspark read file
@Anpan K,After you run the above snippet content is created as an RDD. You can perform operations on that RDD to whatever you want.For ex:%pyspark content = sc.textFile("file:///path/example.txt")...
View ArticleClik here to view.
Re: How to move Spark2 components to another node?
I see Spark2 history server available in my cluster. Can you check if Spark2 history server is already installed on that node.Please refer the screenshotFor clients, move option is not required. You...
View ArticleRe: Spark thrift server is failing to start when NN HA is enabled
@Jay Kumar SenSharma,Thanks for the input. I tried running normal HDFS command and it is working fine even when host1 is on standby. I checked Namenode and ZKFC logs but there is nothing much relevant...
View ArticleRe: Prevent HDFS from creating data directories
@Christos Stefanopoulos,That is the expected behaviour. If you want to achieve this, then you need to create separate configs for each host. For ex: hostA will have only /grid in datanode dirs and...
View ArticleRe: Spark thrift server is failing to start when NN HA is enabled
@Felix Albani,Yes I tried copying both core-site.xml and also hdfs-site.xml but still facing the same issue. Attaching some logs, spark thrift server start logs in debug mode and corresponding yarn...
View ArticleRe: Spark thrift server is failing to start when NN HA is enabled
@Felix Albani,This worked like a charm. Thanks a lot for your help. Really appreciate 🙂However in the latest version of Ambari, it should have been handled by Ambari itself. I do not see the manual...
View ArticleRe: Error running solr query!
@Lok! Reddy,Did you enable Audit to Solr under Ranger Audit sections. Also, I do not see any services in Ranger. Did you restart all the services after enabling ranger plugins.
View ArticleRe: Error running solr query!
Can you try solution mentioned in this questionhttps://community.hortonworks.com/questions/61415/ranger-audit-to-solr-problem.html
View ArticleRe: Spark Learning Clarification
You need not learn both Java and Scala to start with spark. If need to be familiar with either of Java,Scala, Python or R to work with spark. You can start with this tutorial to understand the basics...
View ArticleRe: Spark Learning Clarification
You can work with pyspark if you know python. All the features will be available in pyspark as well. You need not learn scala or java
View ArticleRe: Restart Resource Manager does not work, can not start YARN
@Andreas Kühnert,I guess the directory got deleted somehow from HDFS. You can create it and try starting RM# su hdfs # kinit -kt /etc/security/keytabs/hdfs.headless.keytab {principal} ---> Run this...
View ArticleRe: Ambari access through private hostname on aws issue
@kanna k,Did you make the /etc/hosts entries in your laptop/desktop from where you are accessing the ambari cluster. If not, please add entries in your desktop as well.
View ArticleRe: Ambari access through private hostname on aws issue
/etc/hosts file just translates the hostname to the ip. Even without the mapping you can open the urls by passing the ip. If you want to access it using hostname, then you need the entries in...
View ArticleRe: Restart Resource Manager does not work, can not start YARN
@Andreas Kühnert,Glad that the issue is resolved. Since this is a different issue, I suggest to open a new thread for this issue so that the main thread doesn't get deviated. I'm not sure of that...
View ArticleRe: ATS2-hbase starts but on the wrong node
@Guillaume Roger,I guess ATSv2 is running as a service and not embedded mode. Can you filter for "is_hbase_system_service" in YARN configs and check the value. If it is set to true, then ATS v2 will be...
View ArticleRe: spark 2 jobs with OOZIE on HDP 2.6.5 hangs
@chris herssens,Looks like there is a resource crunch. Can you try adding additional NodeManagers if possible. Check if any applications are running in YARN. You can kill any app if it's not used and...
View ArticleRe: knox PUT request failing
@Sami AhmadFor Knox , you have to make 2 curl calls. The second curl call should be made to the "Location" header obtained from the 1st curl call response.1st curl callcurl -i -k -u...
View ArticleRe: Ranger "Test Connection" button REST API?
@nur majid,You can use the below API to validate the configcurl -u username:password -X POST -d '{post-body-json-mentioned-below}' -H "Content-Type:application/json"...
View Article