Sedeks

Well. Im currently researching how we can test Big Data related applications. After I browse many sites I got some useful information on How we can test BIG Data based Hadoop Systems. BigData testing basically involve Analytics part like BI, ETL testing, running Jobs Flows through any third party Hadoop Implemented Tools. BI: Its purely Report based testing where we will be having Analysed Data Store to Showcase Report based Results through Any third party Reporting Tools like Cognos, MS Bi etc. ETL testing: Traditional ETL tools will be extending capabilty to integrate with Hadoop HDFS( Hadoop Distributed File System).  Ex: In IBM datastage  8.7, There is new sequence stage called BDFS(Big Data File Stage) which will act as connector to Hadoop HDFS. Running MapReduce/Hive Jobs flows: This is pure Hadoop Testing, We will be given an front end tool to configure, Number of instances, The Input file to process, The Output file to store Result, And the Processing Ap

Exception: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf     at java.lang.Class.forName0(Native Method)     at java.lang.Class.forName(Class.java:264)     at org.apache.hadoop.util.RunJar.main(RunJar.java:149) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)     at java.security.AccessController.doPrivileged(Native Method)     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)     at java.lang.ClassLoader.loadClass(ClassLoader.java:321)     at java.lang.ClassLoader.loadClass(ClassLoader.java:266)     ... 3 more Make sure you have below things setup in ./bashrc export HIVE_HOME=/usr/local/hive export PATH=$HIVE_HOME/bin: $PATH   Also Move(or remove) Hadoop_Classpath from Hadoop-env.sh to ./bashrc