scala - How to access local files in Spark on Windows? -


i using spark on windows. know in *nix, accessing local file code likes this: 

val textfile = sc.textfile("file:///usr/local/spark/readme.md")

but how can access local file on windows? have tried following methods: 

val logfile = "c:\spark-1.3.1-bin-hadoop2.4\readme.md" val logfile = "file\\c:\spark-1.3.1-bin-hadoop2.4\readme.md"

but can't work.

unfortunately in windows have escape "\".

try:

"c:\\spark-1.3.1-bin-hadoop2.4\\readme.md"

-

Comments

Post a Comment

Popular posts from this blog

delphi - Disable and change color of node in Treeview -

Image
i looking way of disabling proper nodes . let's have created simple treeview structure below: parent 1 (disable) parent 2 child 1 child 2 (disable) so far have found workaround how that, still doesn't work @ all. use onchanging event handler: procedure tform1.treeviewchanging(sender: tobject; node: ttreenode; var allowchange: boolean); begin if (node.text = 'parent 1') or (node.text = 'child 1') allowchange := false; end; it works, because can't click on node, issue after 'disabling' node way, color of parent 1 , child 1 nodes same other. way change color of nodes (grey)? or maybe know way disabling nodes in ttreeview component? to disable node, set enabled property false . disabled item displayed in gray color. to change normal color of nodes text, change ttreeview.font.color . to change color of disabled nodes text, use oncustomdrawitem event: procedure tform2.treeview1customdrawitem(sender: tcu...
Read more

unity3d - Rotate an object to face an opposite direction -

i have object , b. when click on object b, rotates object b (a faces b). b doesn't face a. need following : when faces b, need face opposite direction. have code rotating @ b. how rotate face opposite direction after that? vector3 targetdirection = target - transform.position; float step = speed * time.deltatime; vector3 newdirection = vector3.rotatetowards (turretdome.transform.forward, targetdirection, step, 0.0f); turretdome.transform.rotation = quaternion.lookrotation (newdirection); you object facing object b, want inverse direction of object after that? objecta.transform.rotation = quaternion.inverse(objecta.transform.rotation) but lets assume example turretdome object a, (negate direction): turretdome.transform.rotation = quaternion.lookrotation (-newdirection); naturally, both of these snippets not show how smoothen rotation, seem know how use time.deltatime. just incase unsure, quaternion.lerp this
Read more

elasticsearch python client - work with many nodes - how to work with sniffer -

i have 1 cluster 2 nodes. i trying understand best practise connect nodes, , check failover when there downtime on 1 node. from documentation : es = elasticsearch( ['esnode1', 'esnode2'], # sniff before doing sniff_on_start=true, # refresh nodes after node fails respond sniff_on_connection_fail=true, # , every 60 seconds sniffer_timeout=60 ) so tried connect nodes this: client = elasticsearch([ip1, ip2],sniff_on_start=true, sniffer_timeout=10,sniff_on_connection_fail=true) where ip1/ip2 machine ip's (for example 10.0.0.1, 10.0.0.2) in order test it, terminated ip2 (or put non existent if) now, when trying connect, get: transporterror: transporterror(n/a, 'unable sniff hosts - no viable hosts found.') even ip1 exist , up. if trying connect this: es = elasticsearch([ip1, ip2]) then can see in log if client not getting response ip2, move ip1, , return valid response. so missing here something? thought s...
Read more