我已經(jīng)編寫了一個(gè)Perl代碼來處理大量CSV文件并獲取輸出,這需要0.8326秒才能完成.

my $opname = $ARGV[0];

my @files = `find . -name "*${opname}*.csv";mtime -10 -type f`;

my %hash;

foreach my $file (@files) {

chomp $file;

my $time = $file;

$time =~ s/.*\~(.*?)\..*/$1/;

open(IN, $file) or print "Can't open $file\n";

while () {

my $line = $_;

chomp $line;

my $severity = (split(",", $line))[6];

next if $severity =~ m/NORMAL/i;

$hash{$time}{$severity}++;

}

close(IN);

}

foreach my $time (sort {$b <=> $a} keys %hash) {

foreach my $severity ( keys %{$hash{$time}} ) {

print $time . ',' . $severity . ',' . $hash{$time}{$severity} . "\n";

}

}

現(xiàn)在,我用Java編寫了相同的邏輯,但我花了2600毫秒(即2.6秒)來完成.我的問題是Java為什么要花這么長(zhǎng)時(shí)間？如何達(dá)到與Perl相同的速度？

注意：我忽略了VM初始化和類加載時(shí)間.

import java.io.BufferedReader;

import java.io.File;

import java.io.FileFilter;

import java.io.FileReader;

import java.io.IOException;

import java.util.HashMap;

import java.util.Map;

import java.util.TreeMap;

public class MonitoringFileReader {

static Map> store= new TreeMap>();

static String opname;

public static void testRead(String filepath) throws IOException

{

File file = new File(filepath);

FileFilter fileFilter= new FileFilter() {

@Override

public boolean accept(File pathname) {

// TODO Auto-generated method stub

int timediffinhr=(int) ((System.currentTimeMillis()-pathname.lastModified())/86400000);

if(timediffinhr<10 && pathname.getName().endsWith(".csv")&& pathname.getName().contains(opname)){

return true;

}

else

return false;

}

};

File[] listoffiles= file.listFiles(fileFilter);

long time= System.currentTimeMillis();

for(File mf:listoffiles){

String timestamp=mf.getName().split("~")[5].replace(".csv", "");

BufferedReader br= new BufferedReader(new FileReader(mf),1024*500);

String line;

Map tmp=store.containsKey(timestamp)?store.get(timestamp):new HashMap();

while((line=br.readLine())!=null)

{

String severity=line.split(",")[6];

if(!severity.equals("NORMAL"))

{

tmp.put(severity, tmp.containsKey(severity)?tmp.get(severity)+1:1);

}

}

store.put(timestamp, tmp);

}

time=System.currentTimeMillis()-time;

System.out.println(time+"ms");

System.out.println(store);

}

public static void main(String[] args) throws IOException

{

opname = args[0];

long time= System.currentTimeMillis();

testRead("./SMF/data/analyser/archive");

time=System.currentTimeMillis()-time;

System.out.println(time+"ms");

}

}

文件輸入格式(A?B?C?D?E?20150715080000.csv),每個(gè)文件約500個(gè),每個(gè)文件約1MB,

A,B,C,D,E,F,CRITICAL,G

A,B,C,D,E,F,NORMAL,G

A,B,C,D,E,F,INFO,G

A,B,C,D,E,F,MEDIUM,G

A,B,C,D,E,F,CRITICAL,G

Java版本：1.7

Update //

根據(jù)以下評(píng)論,

我用regex替換了split,性能得到了很大的改善.

現(xiàn)在,我正在循環(huán)執(zhí)行此操作,經(jīng)過3-10次迭代后,性能還是可以接受的.

import java.io.BufferedReader;

import java.io.File;

import java.io.FileFilter;

import java.io.FileReader;

import java.io.IOException;

import java.util.HashMap;

import java.util.Map;

import java.util.regex.Matcher;

import java.util.regex.Pattern;

public class MonitoringFileReader {

static Map> store= new HashMap>();

static String opname="Etis_Egypt";

static Pattern pattern1=Pattern.compile("(\\d+\\.)");

static Pattern pattern2=Pattern.compile("(?:\"([^\"]*)\"|([^,]*))(?:[,])");

static long currentsystime=System.currentTimeMillis();

public static void testRead(String filepath) throws IOException

{

File file = new File(filepath);

FileFilter fileFilter= new FileFilter() {

@Override

public boolean accept(File pathname) {

// TODO Auto-generated method stub

int timediffinhr=(int) ((currentsystime-pathname.lastModified())/86400000);

if(timediffinhr<10 && pathname.getName().endsWith(".csv")&& pathname.getName().contains(opname)){

return true;

}

else

return false;

}

};

File[] listoffiles= file.listFiles(fileFilter);

long time= System.currentTimeMillis();

for(File mf:listoffiles){

Matcher matcher=pattern1.matcher(mf.getName());

matcher.find();

//String timestamp=mf.getName().split("~")[5].replace(".csv", "");

String timestamp=matcher.group();

BufferedReader br= new BufferedReader(new FileReader(mf));

String line;

Map tmp=store.containsKey(timestamp)?store.get(timestamp):new HashMap();

while((line=br.readLine())!=null)

{

matcher=pattern2.matcher(line);

matcher.find();matcher.find();matcher.find();matcher.find();matcher.find();matcher.find();matcher.find();

//String severity=line.split(",")[6];

String severity=matcher.group();

if(!severity.equals("NORMAL"))

{

tmp.put(severity, tmp.containsKey(severity)?tmp.get(severity)+1:1);

}

}

br.close();

store.put(timestamp, tmp);

}

time=System.currentTimeMillis()-time;

//System.out.println(time+"ms");

//System.out.println(store);

}

public static void main(String[] args) throws IOException

{

//opname = args[0];

for(int i=0;i<20;i++){

long time= System.currentTimeMillis();

testRead("./SMF/data/analyser/archive");

time=System.currentTimeMillis()-time;

System.out.println("Time taken for "+i+" is "+time+"ms");

}

}

}

但是我現(xiàn)在還有一個(gè)問題,

在小型數(shù)據(jù)集上運(yùn)行時(shí)查看結(jié)果.

**Time taken for 0 is 218ms

Time taken for 1 is 134ms

Time taken for 2 is 127ms**

Time taken for 3 is 98ms

Time taken for 4 is 90ms

Time taken for 5 is 77ms

Time taken for 6 is 71ms

Time taken for 7 is 72ms

Time taken for 8 is 62ms

Time taken for 9 is 57ms

Time taken for 10 is 53ms

Time taken for 11 is 58ms

Time taken for 12 is 59ms

Time taken for 13 is 46ms

Time taken for 14 is 44ms

Time taken for 15 is 45ms

Time taken for 16 is 53ms

Time taken for 17 is 45ms

Time taken for 18 is 61ms

Time taken for 19 is 42ms

首先,花費(fèi)的時(shí)間更多,然后減少.

為什么???

謝謝 ,

總結(jié)

以上是生活随笔為你收集整理的perl java_与Perl相比Java性能问题的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。