What is an elegant way to deal with exceptions in map functions in Spark?
For example with:
exampleRDD= ["1","4","7","2","err",3] exampleRDD=exampleRDD.map(lambda x: int(x))
This will not work because it will fail on the "err" item.
How can I filter out faulty rows and execute map on the rest, without anticipating the kind of error that I will encounter in every row?
One could do something like defining a function:
def stringtoint(x): try: a=int(x) except: a=-99 return a
And then filter/map. But this doesn't seem as graceful as could be.