Scala入门到精通——第三十节 Scala脚本编程与结束语
本節主要內容
1. REPL命令行高級使用
在使用REPL命令行時,有時候我們需要粘貼的代碼比較大,而普通的粘貼可能會些一些問題,比如中文粘貼會出現亂碼、多行代碼粘貼時會出錯,此時需要用到REPL的高級功能。在日常開發過程中,我們粘貼多行代碼的時候會遇到下列問題:
//本意是要粘貼下面兩行代碼 class Person(val name:String,val age:Int) val p=new Person("搖擺少年夢",27)//直接在REPL命令行粘貼的話,會出現下面情況 //1 不會一次性粘入,而是分作兩行 //2 中文出現亂碼 scala> class Person(val name:String,val age:Int) defined class Personscala> val p=new Person("??????????",27) p: Person = Person@cf528而對于一些長串跨行的代碼,可能會出現報錯,例如:
//本意是要粘貼下面的代碼 if(p.age>10) trueelsefalse//但實際情況是這樣的 scala> if(p.age>10)| true res0: AnyVal = truescala> else <console>:1: error: illegal start of definitionelse^scala> false那要怎么辦呢?在REPL命令行中執行下列命令:
scala> :paste // Entering paste mode (ctrl-D to finish)if(p.age>10)trueelsefalse// Exiting paste mode, now interpreting.res3: Boolean = true先輸入:paste,然后按ctr+v鍵,可以正常粘貼內容,中文也不會出現亂碼了:
scala> :paste // Entering paste mode (ctrl-D to finish)class Person(val name:String,val age:Int) val p=new Person("搖擺少年夢",27)// Exiting paste mode, now interpreting.defined class Person p: Person = Person@1924b38另外,在實際開發過程中,有些人會認為這種處理方式非常繁瑣,Scala的創建者也為我們考慮過這個問題了,我們也可以在scala IDE for eclipse (在Intellij IDEA 中也有這個功能) 里面利用REPL命令行,使用方式是創建scala worksheet,創建方式如下:
1 點擊相應的包,然后右鍵,在new菜單中選擇 scala worksheet
2 在文件中輸入相應的scala語句,worksheet會自動打印出相應的結果
但是worksheet對中文的支持很不友好,例如下面的代碼:
case class Person(val name:String,val age:Int)object ScalaREPL {println("Scala worksheet") //> Scala worksheetval p=new Person("搖擺少年夢",27) //> p : cn.scala.xtwy.jdbc.Person = Person(鎽囨憜灝戝勾姊�27) }worksheet最終得到的中文是亂碼,因此在實際進行語言特性測試的時候盡量避免中文
scala中還有很多我們實際中沒有接觸過的命令,可以用 :help命令查看REPL現在支持的所有命令:
scala> :help All commands can be abbreviated, e.g. :he instead of :help. Those marked with a * have more detailed help, e.g. :help imports.:cp <path> add a jar or directory to the classpath :help [command] print this summary or command-specific help :history [num] show the history (optional num is commands to show) :h? <string> search the history :imports [name name ...] show import history, identifying sources of names :implicits [-v] show the implicits in scope :javap <path|class> disassemble a file or class name :load <path> load and interpret a Scala file :paste enter paste mode: all input up to ctrl-D compiled tog ether :power enable power user mode :quit exit the interpreter :replay reset execution and replay all previous commands :reset reset the repl to its initial state, forgetting all s ession entries :sh <command line> run a shell command (result is implicitly => List[Str ing]) :silent disable/enable automatic printing of results :type [-v] <expr> display the type of an expression without evaluating it :warnings show the suppressed warnings from the most recent lin e which had any- 1
2. 使用Scala進行Linux腳本編程
本節Linux腳本內容大部分來源于scala cookbook,部分經過本人修改以在Ubuntu linux上進行演示。
我們在第一節中提到,Scala不僅僅可以進行大規模分布式應用程序開發(例如Spark內存計算框架),也可以進行服務器端腳本編程即它可以替代Linux中的shell (Bourne Shell, Bash)或其它如 Perl, PHP, Ruby等可用于服務器端腳本編程的語言。下面給出的是一個簡單示例(前提是要有linux操作系統,本節所有示例都是在ubuntu Linux下運行的):
#!/bin/sh exec scala "$0" "$@" !# println("HellO,Linux World")將上面的內容保存為hello.sh文件,然后用下列命令增加其執行權限:
root@sparkmaster:/home/zhouzhihu/scalaLearning# chmod +x hello.sh root@sparkmaster:/home/zhouzhihu/scalaLearning# ./hello.sh HellO,Linux World可以看到我們第一個服務器腳本已經運行成功。前面的代碼中,#!符號表示的是Unix shell腳本的開始,它會調用Unix Bourne shell。exce命令是內置的shell,表示需要執行scala 命令,其中0綁定的是hello.sh腳本名稱,@ 綁定的是我們輸入的參數。!#表示腳本聲明頭部的結束。在腳本中可以使用任何的scala語法,例如:
#!/bin/sh exec scala "$0" "$@" !#class Person(val firstName:String,val secondName:String){override toString()="firstName:"+firstName+",secondName:"+secondName }println(new Person("john","Barake"))上述代碼執行結果:
root@sparkmaster:/home/zhouzhihu/scalaLearning# ./person.sh firstName:john,secondName:Barake除此之外,我們還可以定義應用程序對象,可以擴展自App,也可以實現自己的Main方法,例如:
#!/bin/sh exec scala "$0" "$@" !# object Hello extends App { println("Hello Ubuntu Linux 10.04") //如果后面帶參數的話,可以捕獲所有的參數 args.foreach(println) } Hello.main(args)下面給出的是不帶參數的執行結果:
root@sparkmaster:/home/zhouzhihu/scalaLearning# ./HelloApp.sh Hello Ubuntu Linux 10.04下面給出的是帶參數的執行結果,如:
root@sparkmaster:/home/zhouzhihu/scalaLearning# ./HelloApp.sh hello xuetuwuyouHello Ubuntu Linux 10.04 hello xuetuwuyou當然,還可以實現自己的main方法,如:
#!/bin/sh exec scala "$0" "$@" !# object Hello {def main(args: Array[String]) {println("Hello, world")args.foreach(println)} } Hello.main(args)同extends App是一樣的。
如果腳本中需要應用到第三方庫的話,可以采用下列方式進行包引入:
#!/bin/sh exec scala -classpath "lib/slick_2.11_2.1.0.jar:lib/mysql-connector-java-5.1.18-bin.jar" "$0" "$@" !#import scala.slick.driver.MySQLDriver.simple._object CoffeeExample extends App {class Suppliers(tag: Tag) extends Table[(Int, String, String, String, String, String)](tag, "SUPPLIERS") {def id = column[Int]("SUP_ID", O.PrimaryKey) // This is the primary key columndef name = column[String]("SUP_NAME")def street = column[String]("STREET")def city = column[String]("CITY")def state = column[String]("STATE")def zip = column[String]("ZIP")// Every table needs a * projection with the same type as the table's type parameterdef * = (id, name, street, city, state, zip)}val suppliers = TableQuery[Suppliers]// Definition of the COFFEES tableclass Coffees(tag: Tag) extends Table[(String, Int, Double, Int, Int)](tag, "COFFEES") {def name = column[String]("COF_NAME", O.PrimaryKey)def supID = column[Int]("SUP_ID")def price = column[Double]("PRICE")def sales = column[Int]("SALES")def total = column[Int]("TOTAL")def * = (name, supID, price, sales, total)// A reified foreign key relation that can be navigated to create a joindef supplier = foreignKey("SUP_FK", supID, suppliers)(_.id)}val coffees = TableQuery[Coffees]Database.forURL("jdbc:mysql://localhost:3306/slick", "root", "123",driver = "com.mysql.jdbc.Driver") withSession {implicit session =>// Create the tables, including primary and foreign keys(suppliers.ddl ++ coffees.ddl).create// Insert some supplierssuppliers += (101, "Acme, Inc.", "99 Market Street", "Groundsville", "CA", "95199")suppliers += (49, "Superior Coffee", "1 Party Place", "Mendocino", "CA", "95460")suppliers += (150, "The High Ground", "100 Coffee Lane", "Meadows", "CA", "93966")// Insert some coffees (using JDBC's batch insert feature, if supported by the DB)coffees ++= Seq(("Colombian", 101, 7.99, 0, 0),("French_Roast", 49, 8.99, 0, 0),("Espresso", 150, 9.99, 0, 0),("Colombian_Decaf", 101, 8.99, 0, 0),("French_Roast_Decaf", 49, 9.99, 0, 0))coffees foreach {case (name, supID, price, sales, total) =>println(" " + name + "\t" + supID + "\t" + price + "\t" + sales + "\t" + total)}val q1 = for (c <- coffees)yield LiteralColumn(" ") ++ c.name ++ "\t" ++ c.supID.asColumnOf[String] ++"\t" ++ c.price.asColumnOf[String] ++ "\t" ++ c.sales.asColumnOf[String] ++"\t" ++ c.total.asColumnOf[String]// The first string constant needs to be lifted manually to a LiteralColumn// so that the proper ++ operator is foundq1 foreach println// Perform a join to retrieve coffee names and supplier names for// all coffees costing less than $9.00val q2 = for {c <- coffees if c.price < 9.0s <- suppliers if s.id === c.supID} yield (c.name, s.name)}} //這點與一般的應用程序不同 CoffeeExample.main(args)通過上述代碼不難發現,腳本編程與一般的Scala應用程序開發有著非常多的相似之處,不同之處僅在于在腳本編程需要加入下面這樣的樣板代碼
#!/bin/sh //樣板代碼 exec scala -classpath "lib/slick_2.11_2.1.0.jar:lib/mysql-connector-java-5.1.18-bin.jar" "$0" "$@" !# //樣板代碼....................CoffeeExample.main(args) //樣板代碼有時候,我們也需要對命令行參數進行捕獲(例如判斷命令行的個數或輸入的參數類型等),然后進行相應的操作,前面已經演示了如何打印輸出命令行參數,這里我們更多實際中可能會遇到的一些經典案例:
1 判斷輸入參數的個數,不滿足要求則給出提示
執行結果如下:
root@sparkmaster:/home/zhouzhihu/scalaLearning# ./argsNumberDemo.sh xuetuwu xuetuwuyou Replacing xuetuwu with xuetuwuyou ... root@sparkmaster:/home/zhouzhihu/scalaLearning# ./argsNumberDemo.sh Usage: replacer <search> <replace>2 交互式命令行,提示用戶輸入
#!/bin/sh exec scala "$0" "$@" !# // write some text out to the user with Console.println Console.println("Hello") // Console is imported by default, so it's not really needed, just use println println("World") // readLine lets you prompt the user and read their input as a String val name = readLine("What's your name? ") // readInt lets you read an Int, but you have to prompt the user manually print("How old are you? ") val age = readInt() // you can also print output with printf println(s"Your name is $name and you are $age years old.")下面給出的是其執行結果:
root@sparkmaster:/home/zhouzhihu/scalaLearning# ./promptDemo.sh Hello World What's your name? yaobaishaonianmeng How old are you? 27 Your name is yaobaishaonianmeng and you are 27 years old.- 1
3 加速代碼的執行:
scala腳本在執行的過程中,也是通過編譯、執行的步驟來進行的,有時候為加速腳本的執行,意圖是將編譯后的腳本保存下來,在執行時候如果腳本創建之后沒有發生變化的話,則直接使用以前編譯好的腳本。實現方式是在腳本聲明的時候用-savecompiled。
它的原理是在代碼第一次執行后,生成相應的jar文件,當下次再執行的便調用該jar文件來執行,第一次執行后生成的jar文件如下:
3. 結束語
這門課程的目的是讓大家學完之后能夠快速上手spark應用程序開發,希望在學完本課程之后,大家將這門課作為自己學習scala的起點,而非終點。
總結
以上是生活随笔為你收集整理的Scala入门到精通——第三十节 Scala脚本编程与结束语的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: JVM性能调优中的命令总结
- 下一篇: Spark Shuffle Write阶