Instead of using a DAG plan generated by TEZ in Hive, I want to use custom DAG plan created by me. Is it Possible? If Yes, can anyone explain?
Can we impose custom DAG plan over generated DAG plan in TEZ? If possible, How?
89 views Asked by Ramprakash At
1
There are 1 answers
Related Questions in HIVE
- Type Adapter for Offset in hive flutter
- HIVE Sql Date conversion
- How to set spark.executor.extraClassPath & spark.driver.extraClassPath in hive query without adding those in hive-site.xml
- Hive query on HUE shows different timestamp than programatically/on data
- descending order of data in hive using collect_set
- How to optimize writing to a large table in Hive/HDFS using Spark
- Spark SQL repartition before insert operation
- Alter datatype of complex type(array<struct>>) in hive
- SqlAlchemy connection to Hive using http thrift transport and basic auth
- Aggregate values into a new column while retaining the old column
- Is it possible to query MAPR hdfs/hive tables from Trino?
- Can we make a column having both partitioning and bucketing in hive?
- converting varchar(7) to decimal (7,5) in hive
- Extract all characters before numeric values in hive SQL
- Livy session to submit pyspark from HDFS
Related Questions in DIRECTED-ACYCLIC-GRAPHS
- I want to monitor a job triggered through emrserverlessstartjoboperator. If the job is either is success or failed, want to rerun the job in airflow
- airflow dags not running as expected
- Create a daily DAG that will run for multiple days
- Airflow config for running concurrent DAG tasks
- Algorithm for total flow through weighted directed acyclic graph
- Scalable Python Shortest Path on Large DAG with Multiple Walkers
- Finding path with smallest GCD of nodes's weights in directed graph
- How to Stop an Airflow DAG and Skip Future Processing if File is Empty?
- Is there a way to test looser airflow DAG dependencies? i.e. not direct dependencies
- Can multiprocessing in Python be used to simulate a graph evolving through time?
- Airflow Parallel DAG runs
- How perform task in airflow that executes regardless of failed tasks but fail the DAG when error appeared in previous tasks?
- Create custom Apache Airflow FileSensor that retries after sensor timeout
- Cost for using GoogleCloudStorageToBigQueryOperator in Airflow
- How can I successfully start the apache airflow db
Related Questions in TEZ
- Preemption with Tez along with the yarn FairShare scheduler supported?
- Way to get current editor id (doc id) in HiveServerClient in Hue source code
- Hive’s dynamic partitioning failing to write final files
- Hive complex data type querying
- Failed Vertex re-running
- How many number of reducers will be running if I use distinct on all columns in hive?
- Poor performance on hash joins with Pig on Tez
- Hive - Select count(*) not working with Tez with but works with MR
- Hive Vertex failed, vertexName=Map
- Hive + Tez :: A join query stuck at last 2 mappers for a long time
- hive llap - which execution engine supported? spark,mr, tez
- Hive - How to know which execution engine I am currently using
- Hadoop Hive TEZ task fails (to start container)
- Pig script on aws emr with tez occasionally fails with OutOfMemoryException
- OOM in tez/hive
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
If you are using Hive for running queries then you cannot use custom DAG. If you do low level programming Tez application using API, you can. See High level docs here: https://hortonworks.com/blog/expressing-data-processing-in-apache-tez/
Something like this:
See apache project examples: https://git-wip-us.apache.org/repos/asf?p=tez.git;a=blob;f=tez-examples/src/main/java/org/apache/tez/examples/HashJoinExample.java