I want to Write the dynamic frame into my redshift, but i got error that says
Unrecognized scheme null; expected s3, s3n, or s3a
i have tried like this below.
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)
datasource0 = glueContext.create_dynamic_frame.from_catalog(database = "data_input", table_name = "db_public_trx", transformation_ctx = "datasource0")
datasource1 = glueContext.create_dynamic_frame.from_catalog(database = "data_output", table_name = "redshiftdev_public_trx", transformation_ctx = "datasource1")
src_df = datasource0.toDF()
dst_df = datasource1.toDF()
merged_df = dst_df.union(src_df)
datasource3 = fromDF(merged_df,glueContext,"datasource3")
datasink = glueContext.write_dynamic_frame.from_catalog(frame = datasource3, database = "data_output", table_name = "redshiftdev_public_trx", redshift_tmp_dir = args["TempDir"], transformation_ctx = "datasink")
job.commit()
Copyright Notice:Content Author:「Muhammad Ariq Naufal」,Reproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/69386295/illegalargumentexception-unrecognized-scheme-null-expected-s3-s3n-or-s3a