Delta Live Table - Cannot redefine dataset

31 views Asked by At

I am new to Delta Live Table.

I am trying to create a delta live table from the databricks tutorial.

I have created a notebook and attached an interactive cluster -DBR 14.3-LTS.

I am running the below code.

When I ran it for the 1st time it ran successfully.

When I ran the cell for the 2nd time I am getting error - AnalysisException: Cannot redefine dataset 'sales_orders_raw'

Can you please help me understand why this is happening?


from pyspark.sql.functions import *
from pyspark.sql.types import *
import dlt

@Dlt.create_table(
  comment="The raw sales orders, ingested from /databricks-datasets.",
  table_properties={
    "myCompanyPipeline.quality": "bronze",
    "pipelines.autoOptimize.managed": "true"
  }
)
def sales_orders_raw():
  return (
    spark.readStream.format("cloudFiles") \
      .option("cloudFiles.schemaLocation", "/tmp/john.odwyer/pythonsalestest") \
      .option("cloudFiles.format", "json") \
      .option("cloudFiles.inferColumnTypes", "true") \
      .load("/databricks-datasets/retail-org/sales_orders/")
  )

Can you please help me understand what I am doing wrong

0

There are 0 answers