Reading a CSV file in spark

on
Reading CSV files in spark:

We need few options to load csv files:

delimiter : Separator between each file, for example (comma, pipe, tab).
header : True if file has header, False when file doesn't have header.
inferSchema : Spark can intelligently read the data in file and predict data type of each column


Input file data:



id,name,marks
1,Shiva,90
2,Ram,85
3,Mohan,95
4,Raju,96


Source code:
df = spark.read.option("delimiter",", ").option("header","true").option("inferSchema","true").csv("C:\\demo\\files\\cities.csv")

df.show()



Output:


Comments

Post a Comment