Cyber-Physical Systems: A Case for Soft Real-Time, Ulrich Kremer
http://www.research.rutgers.edu/~uli/Sarana/documents/CPS-Uli.pdf
gnuplotでeps
14 年前
2011年4月10日日曜日
[IOT] IEEE/ACM International Conference on Internet of Things 2011
http://www.linggantek.com/IoT2011/IOT2011%20CFP.pdf
締切りは 5月1日。
トラックは以下の通り。
• Track 1: Architecture and Infrastructure
• Track 2: System Design, Modeling and Evaluation
• Track 3: Intelligent Data Processing
• Track 4: Networks and Communications
• Track 5: Reliability, Security, Privacy and Trust, chaired by Isaac
• Track 6: Applications, Business and Social Issue
締切りは 5月1日。
トラックは以下の通り。
• Track 1: Architecture and Infrastructure
• Track 2: System Design, Modeling and Evaluation
• Track 3: Intelligent Data Processing
• Track 4: Networks and Communications
• Track 5: Reliability, Security, Privacy and Trust, chaired by Isaac
• Track 6: Applications, Business and Social Issue
Streaming Particle Filter
Real-Time Particle Filter, 2004
http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.72.657
Particle filters estimate the state of dynamical systems from sensor information. In many real time applications of particle filters, however, sensor information arrives at a significantly higher rate than the update rate of the filter. The prevalent approach to dealing with such situations is to update the particle filter as often as possible and to discard sensor information that cannot be processed in time. In this paper we present real-time particle filters, which make use of all sensor information even when the filter update rate is below the update rate of the sensors. This is achieved by representing posteriors as mixtures of sample sets, where each mixture component integrates one observation arriving during a filter update. The weights of the mixture components are set so as to minimize the approximation error introduced by the mixture representation. Thereby, our approach focuses computational resources (samples) on valuable sensor information. Experiments using data collected with a mobile robot show that our approach yields strong improvements over other approaches. 1
http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.72.657
Particle filters estimate the state of dynamical systems from sensor information. In many real time applications of particle filters, however, sensor information arrives at a significantly higher rate than the update rate of the filter. The prevalent approach to dealing with such situations is to update the particle filter as often as possible and to discard sensor information that cannot be processed in time. In this paper we present real-time particle filters, which make use of all sensor information even when the filter update rate is below the update rate of the sensors. This is achieved by representing posteriors as mixtures of sample sets, where each mixture component integrates one observation arriving during a filter update. The weights of the mixture components are set so as to minimize the approximation error introduced by the mixture representation. Thereby, our approach focuses computational resources (samples) on valuable sensor information. Experiments using data collected with a mobile robot show that our approach yields strong improvements over other approaches. 1
[StreamGraph] Kronecker Graph
Graph500 にも採用されいているが、実グラフの近似モデルとして最近注目されている Kronecker Graph
http://www.graphanalysis.org/SIAM-PP08/Leskovic.pdf
http://www.cs.cmu.edu/~jure/pubs/kronFit-icml07.pdf
http://arxiv.org/abs/0812.4905
http://www.graphanalysis.org/SIAM-PP08/Leskovic.pdf
http://www.cs.cmu.edu/~jure/pubs/kronFit-icml07.pdf
http://arxiv.org/abs/0812.4905
[StreamGraph] Massive Streaming Data Analytics: A Case Study with Clustering Coefficients
D. Ediger, K. Jiang, J. Riedy, and D.A. Bader, ``Massive Streaming Data Analytics: A Case Study with Clustering Coefficients,'' 4th Workshop on Multithreaded Architectures and Applications (MTAAP), Atlanta, GA, April 23, 2010.
http://www.cc.gatech.edu/~bader/papers/StreamingCC.html
We present a new approach for parallel massive graph analysis of streaming, temporal data with a dynamic and extensible representation. Handling the constant stream of new data from health care, security, business, and social network applications requires new algorithms and data structures. We examine data structure and algorithm trade-offs that extract the parallelism necessary for high-performance updating analysis of massive graphs. Static analysis kernels often rely on storing input data in a specific structure. Maintaining these structures for each possible kernel with high data rates incurs a significant performance cost. A case study computing clustering coefficients on a general-purpose data structure demonstrates incremental updates can be more efficient than global recomputation. Within this kernel, we compare three methods for dynamically updating local clustering coefficients: a brute-force local recalculation, a sorting algorithm, and our new approximation method using a Bloom filter. On 32 processors of a Cray XMT with a synthetic scale-free graph of 224 ≈ 16 million vertices and 229 ≈ 537 million edges, the brute-force method processes a mean of over 50,000 updates per second and our Bloom filter approaches 200,000 updates per second.
http://www.cc.gatech.edu/~bader/papers/StreamingCC.html
We present a new approach for parallel massive graph analysis of streaming, temporal data with a dynamic and extensible representation. Handling the constant stream of new data from health care, security, business, and social network applications requires new algorithms and data structures. We examine data structure and algorithm trade-offs that extract the parallelism necessary for high-performance updating analysis of massive graphs. Static analysis kernels often rely on storing input data in a specific structure. Maintaining these structures for each possible kernel with high data rates incurs a significant performance cost. A case study computing clustering coefficients on a general-purpose data structure demonstrates incremental updates can be more efficient than global recomputation. Within this kernel, we compare three methods for dynamically updating local clustering coefficients: a brute-force local recalculation, a sorting algorithm, and our new approximation method using a Bloom filter. On 32 processors of a Cray XMT with a synthetic scale-free graph of 224 ≈ 16 million vertices and 229 ≈ 537 million edges, the brute-force method processes a mean of over 50,000 updates per second and our Bloom filter approaches 200,000 updates per second.
ストリーム処理用クエリ言語 CQL と StreamSQL の比較
ストリーム処理用クエリ言語 CQL と StreamSQL の比較
http://www.it.uu.se/research/group/udbl/Theses/RobertKajicBSc.pdf
http://www.it.uu.se/research/group/udbl/Theses/RobertKajicBSc.pdf
[Graph500] An effective GPU implementation of breadth-first search
An effective GPU implementation of breadth-first search
Breadth-first search (BFS) has wide applications in electronic design automation (EDA) as well as in other fields. Researchers have tried to accelerate BFS on the GPU, but the two published works are both asymptotically slower than the fastest CPU implementation. In this paper, we present a new GPU implementation of BFS that uses a hierarchical queue management technique and a three-layer kernel arrangement strategy. It guarantees the same computational complexity as the fastest sequential version and can achieve up to 10 times speedup.
Costs of DNA Sequencing Falling Fast
Costs of DNA Sequencing Falling Fast
http://singularityhub.com/2011/03/05/costs-of-dna-sequencing-falling-fast-look-at-these-graphs/
http://singularityhub.com/2011/03/05/costs-of-dna-sequencing-falling-fast-look-at-these-graphs/
[IOT] Internet of Things Survey
Internet of Things: A Survey
Luigi Atzori, Antonio Iera, and Giacomo Morabito. 2010. The Internet of Things: A survey. Comput. Netw. 54, 15 (October 2010), 2787-2805. DOI=10.1016/j.comnet.2010.05.010 http://dx.doi.org/10.1016/j.comnet.2010.05.010
This paper addresses the Internet of Things. Main enabling factor of this promising paradigm is the integration of several technologies and communications solutions. Identification and tracking technologies, wired and wireless sensor and actuator networks, enhanced communication protocols (shared with the Next Generation Internet), and distributed intelligence for smart objects are just the most relevant. As one can easily imagine, any serious contribution to the advance of the Internet of Things must necessarily be the result of synergetic activities conducted in different fields of knowledge, such as telecommunications, informatics, electronics and social science. In such a complex scenario, this survey is directed to those who want to approach this complex discipline and contribute to its development. Different visions of this Internet of Things paradigm are reported and enabling technologies reviewed. What emerges is that still major issues shall be faced by the research community. The most relevant among them are addressed in details.
2011年4月9日土曜日
[ElasticStream] システム構成と実験方法
以下石井君の ElasticStreamシステムの構成と実験方法です。
//---------------------------------------------------------------------------------
// ElasticStream システム 構成と実験方法
// 2011/04/08 石井
//---------------------------------------------------------------------------------
- se01 を esther から wake-on-lan で起動 --> /usr/local/bin/wol.py se01
- se01 (このノードだけ外部からのコネクションを受け付ける。萩原さんにお願いしている。有効期限は2011年3月まで）を wake-on-lan で起動
<初期ディレクトリ構成>
ElasticStreamSystem
-elasticStream //ローカルに配置する設定ファイル群。今までは/tmp/elasticStream の様に配置
-spade //SPADE用プロジェクトファイル
-ec2 //Amazon EC2に関するデータ、及びRubyスクリプト群。
上記ディレクトリをコピー。
ec2/*.id の権限は注意
<ノード定義>
se01: *.dpsを実行するノード。、ResultGetterもこのノードで起動
st系など: 計算ノードとして使う。
<事前準備>
・elasticStreamフォルダ(入力ファイル、出力ファイル、設定ファイル）を /tmp/elasticStream のようにローカル上に配置。
・spade/data/result -> <ローカル上に配置したelasticStreamディレクトリへのパス>になるようにリンクを設定。
このような感じで、.bashrcにパスを通しておく
目的はEC2-API-toolへのパス、鍵・証明書の場所、デフォルトのリージョンの場所など
//---------------------------------------------------------------------------------
以下の環境変数を .bashrc に追加
export EC2_HOME=$HOME/ec2/ec2-api-tools
export PATH=$PATH:$JAVA_HOME/bin:$EC2_HOME/bin:/usr/local/cula/bin:/usr/local/cuda/bin
export EC2_PRIVATE_KEY=$HOME/ec2/pk-D4I5F3YSJGEX46MXYBRSGAEWPZTIF5BD.pem
export EC2_CERT=$HOME/ec2/cert-D4I5F3YSJGEX46MXYBRSGAEWPZTIF5BD.pem
export EC2_URL="https://ec2.us-west-1.amazonaws.com" ### デフォルトリージョン。東京のデータセンターに書き換えるときは Amazon EC2 のサイトを参照。明示的に API でリージョンを指定する必要がある。現在は Ruby スクリプトに書いている。
// ※鍵は同梱のec2フォルダに入っているのでそれへのパスを与える。
// デフォルトのリージョンはこの場合はUS-West
//---------------------------------------------------------------------------------
<設定ファイル解説>
//---------------------------------------------------------------------------------
sla.dat (/tmp/elasticStream/input/に存在。他の設定ファイルも同様の場所に存在)
//---------------------------------------------------------------------------------
動作の設定ファイル
[1行目]
・SLAとなるレイテンシ（マイクロ秒）（今回は使わない）
[2行目]
・TimeSlotの時間幅
[3行目]
・スタンドアローンモードで起動するか否か（1:スタンドアローン, 0:非スタンドアローン）
※スタンドアローンモードではVM Managerを起動せず、クラウド環境を利用しなくなる。
//---------------------------------------------------------------------------------
node_[local/cloud]_*.dat
//---------------------------------------------------------------------------------
計算資源のホスト名のリスト（インスタンスタイプのカテゴリ毎にある）
各行がホスト名。
#でコメントアウト可能。
*** あらかじめ EC2 上でノードを立ち上げている場合には、node_cloud にそのホスト名を指定
//---------------------------------------------------------------------------------
setting.dat
//---------------------------------------------------------------------------------
各種設定ファイル。
[1行目]
・TimeSlotの時間幅（予想済みデータレートを与えるので使用しない）
[2行目]
・データレートの時系列変化を表す入力ファイル名（ファイルの中は1秒ごとのタプル送信数のcsv）
[3行目]
・入力ファイル名
[4〜8行目]
・データレートの振り分け比率の初期値
（インスタンスタイプのカテゴリ毎: local_1, local_2, Cloud_1, Cloud_2, Cloud_3 の順番）
*** 前準備
- コンソールを４つ（？）起動 (３つは se01, １つは st07 (ローカルの計算ノード）)
//---------------------------------------------------------------------------------
<起動方法>
・ローカルの各計算ノードでElasticStreamSystem/ec2/4regex.rb（Stableな計算スクリプト）を実行（引数は0（↑のインスタンスタイプのカテゴリの番号）, 5000（内部の負荷forループの周回数））
※0,5000は以前の実験で使った値なので基本的には0,5000を引数として与えれば良いです。
# ruby 4regex.rb 0 5000
(ポート番号 10101 でコネクションを待っている状態になる）
・システム起動ノードでresultGetter.rbを起動（引数は10101(入力ポート), 12000(出力ポート, System S の Source (client) オペレータに渡すポート)）
# ruby resultGetter.rb 10101 12000
** System S が起動するのを待つ
・deus.rbを起動（引数なし）（各TimeSlotの平均データレートを与えるスクリプト。FutureDetectionの代わり）
# ruby deus.rb
System S の Optimizer オ (UDOP) の前の Source のポート番号 11111 に接続
・xVMManager.rbを起動（引数なし）（VM管理スクリプト）
# ruby xVMManager.rb
・以上のすべての起動を確認したら、submit_job*.shでSystem Sのジョブを起動
※クラウド環境との通信は、xVMManager.rbが自動で行ってくれる（はず）（2011/04/08時点では未テスト）
** ./start_streams.....sh を最初に起動
** ./canceljob__.sh
** ５分経過するとクラウドにつなぐ
** Amazon EC2 のVMインスタンスはマニュアルで停止する必要があるときがあるので注意
・終了する場合は、cancel_job*.shでSystem Sのジョブを止めた後、残りのRubyスクリプトを落とす
//---------------------------------------------------------------------------------
<実験結果>
・結果はすべて、<ローカルのelasticStreamディレクトリ>/data 以下に出力される
//---------------------------------------------------------------------------------
DataRate.csv
//---------------------------------------------------------------------------------
System S側で測定したデータレート
//---------------------------------------------------------------------------------
FutureDetection.csv
//---------------------------------------------------------------------------------
FutureDetection コンポーネント（現在はSDARアルゴリズム）の出力
//---------------------------------------------------------------------------------
Latency_*****.csv
//---------------------------------------------------------------------------------
レイテンシの実測値。各タプルのレイテンシの、1秒ごとの平均。
ALLはすべてのタプル、Local_*, Cloud_*はローカル計算資源、クラウド資源ごとの平均。
実験データとしてはALLを用いた。
//---------------------------------------------------------------------------------
Log_***.csv
//---------------------------------------------------------------------------------
各コンポーネントのログ出力。
エラー出力などもここに出力される。
//---------------------------------------------------------------------------------
Result.csv
//---------------------------------------------------------------------------------
正規表現マッチングでヒットしたタプル（JSONファイル）の"text"要素の出力
正常動作の確認用。
// SSH の設定 - パスフレーズの設定
.ssh/config というファイルを作り、以下の内容を書き足す
host *
StrictHostKeyChecking no
// AWS 管理コンソール
suzucommon@gmail.com /
Region を US West に指定
インスタンス：
stop（使い回す, pending/resume、EBSから消去しない. ただ EBSのストレージ使用量はかかる) ではなく、terminate(EBSからVMインスタンスを消去する）の状態にする.
./cancel したら terminate する。
2011年4月7日木曜日
Graph500：ペタレベルの大規模グラフを解く
2010年10月で発表された Graph500 のリストでは、頂点サイズが 140TB, 1.1PB の問題に関しては解けていない。メインメモリに収まりきるほどのサイズではないので、当然、ディスク (SCM も含めて）も活用しなければいけない。
「基本的な戦略は多段階。1段階目ではメモリに載りきるだけ代表点だけを抜き出して、そのレベルで最短経路探索を行う。2段階目では1段階で求まった最短経路の各辺の中での最短経路を求める。2段階目でもメモリに載りきらない場合には更に上記のステップを繰り返す」
上記アルゴリズムの他に、「メモリ-SSD-HDD-分散ファイルシステム」を透過的に見せるデータストアの設計・開発が必要。
2011年4月6日水曜日
IPDPS 2011 Program
今年5月16日から20日まで US で開催される IPDPS 2011 のプログラムが以下に掲載されました。
http://www.ipdps.org/ipdps2011/2011_advance_program.html非常に興味深い論文が多いのですが、公開されたら以下の論文を読みましょう。
- Multi-GPU MapReduce on GPU Clusters
- X10 as a parallel language for scientific computation: practice and experience
- Communication Optimizations for Distributed-Memory X10 Programs
- Graph Partitioning with Natural Cuts
- A Scalable and Elastic Publish/Subscribe Service
- CATCH: A Cloud-based Adaptive Data Transfer Service for HPC
- DryadOpt: Branch-and-Bound on Distributed Data-Parallel Execution Engines
- Profiling Heterogeneous Multi-GPU Systems to Accelerate Cortically Inspired Learning Algorithms
- PHAST: Hardware-Accelerated Shortest Path Trees
- Using Shared Memory to Accelerate MapReduce on Graphics Processing Units
- Computing Strongly Connected Components in Parallel on CUDA
- Fast Community Detection Algorithm With GPUs and Multi-core Architectures
- The Weighted Byzantine Agreement Problem
- Automated architecture-aware mapping of streaming applications onto GPUs
- Tolerant Value Speculation in Coarse-Grain Streaming Computations
- A Performance and Area Efficient Architecture for Intrusion Detection Systems
- Communication-Avoiding QR Decomposition for GPUs
- Architectural constraints to attain 1 Exaflop/s on three scientific application classes
2011年4月4日月曜日
ストリームコンピューティングのデバッグ環境
Visual debugging for stream processing applications
http://portal.acm.org/citation.cfm?id=1939403&preflayout=tabs
http://portal.acm.org/citation.cfm?id=1939403&preflayout=tabs
2011年4月2日土曜日
[StreamCloud] IEEE Cloud 2011
石井君の論文が IEEE CLOUD 2011 (The 4th International Conference on Cloud Computing) に採択されました。おめでとうございます！採択率は 36/196 (18%) でした。
2011年3月30日水曜日
近況
さてそろそろ新年度に入ります。石井君の SACSIS 2011 に関してはまとまりつつありますが、、実験に関しては再現できるようにドキュメントを残しておいてください。ACS論文誌に関しては、雁瀬君・上野君が鋭意実装・執筆中です。頑張りましょう。
研究室の節電対策、仮想マシンを使ったサーバーの集約化に関しては、上記論文が落ち着いたら考えていきましょう。
インターンに関しては、France INRIA, Tennessee (米国HPC界を牛耳る Jack Dongarra教授がリード), GPMR (MapReduceのマルチGPUを実装したチーム) にコンタクトを取りました。連絡を待ちましょう。また、SanDiego SuperComputer Center, IBM Watson, なども考えています。本人同士の面識があるとベターですが、反応を見ながら一つずつコンタクトを取っていきたいと思います。IBM TRL にしろ、海外のインターンに関して紹介するというのも当たり間なことだと思わないでください。確かな信頼を得た人、結果を出した人がそのような機会が得られるのは当然のことです。
2011年3月24日木曜日
[MapReduce] Mars マルチGPU/マルチノード化・関連論文
この分野（特にGPU関連）はスピード勝負です。特に以下の IPDPS 2011 (5月) の論文は必見。
Multi-GPU MapReduce on GPU Clusters, IPDPS 2011
http://www.idav.ucdavis.edu/publications/print_pub?pub_id=1051
We present GPMR, our MapReduce library that leverages the power of GPU clusters for large-scale computing. To better utilize the GPU, we modify MapReduce by combining large amounts of map and reduce items into chunks and using partial reductions and accumulation. We use persistent map and reduce tasks and stress aspects of GPMR with a set of standard MapReduce benchmarks. We run these benchmarks on a GPU cluster and achieve desirable speedup and efficiency for all benchmarks. We compare our implementation to the current-best GPU-MapReduce library (runs only on a solo GPU) and a highly-optimized multi-core MapReduce to show the power of GPMR. We demonstrate how typical MapReduce tasks are easily modified to fit into GPMR and leverage a GPU cluster. We highlight how total and relative amounts of communication affect GPMR. We conclude with an exposition on the types of MapReduce tasks well-suited to GPMR, and why some tasks need more modifications than others to work well with GPMR.
Designing efficient sorting algorithms for manycore GPUs, IPDPS 2009
http://portal.acm.org/citation.cfm?id=1587667
Tiled-MapReduce: optimizing resource usages of data-parallel applications on multicore with tiling, PACT 2010,
http://portal.acm.org/citation.cfm?id=1854337&CFID=14942240&CFTOKEN=86587012
Multi-GPU MapReduce on GPU Clusters, IPDPS 2011
http://www.idav.ucdavis.edu/publications/print_pub?pub_id=1051
We present GPMR, our MapReduce library that leverages the power of GPU clusters for large-scale computing. To better utilize the GPU, we modify MapReduce by combining large amounts of map and reduce items into chunks and using partial reductions and accumulation. We use persistent map and reduce tasks and stress aspects of GPMR with a set of standard MapReduce benchmarks. We run these benchmarks on a GPU cluster and achieve desirable speedup and efficiency for all benchmarks. We compare our implementation to the current-best GPU-MapReduce library (runs only on a solo GPU) and a highly-optimized multi-core MapReduce to show the power of GPMR. We demonstrate how typical MapReduce tasks are easily modified to fit into GPMR and leverage a GPU cluster. We highlight how total and relative amounts of communication affect GPMR. We conclude with an exposition on the types of MapReduce tasks well-suited to GPMR, and why some tasks need more modifications than others to work well with GPMR.
Designing efficient sorting algorithms for manycore GPUs, IPDPS 2009
http://portal.acm.org/citation.cfm?id=1587667
Tiled-MapReduce: optimizing resource usages of data-parallel applications on multicore with tiling, PACT 2010,
http://portal.acm.org/citation.cfm?id=1854337&CFID=14942240&CFTOKEN=86587012
2011年3月16日水曜日
2011年3月12日土曜日
[StreamCloud] 関連論文
Balancing Load in Stream Processing with the Cloud
http://lsds.doc.ic.ac.uk/research/doc/smdb11-cloud_streaming.pdf
Stream processing systems must handle stream data coming from real-time, high-throughput applications, for example in financial trading. Timely processing of streams is important and requires sufficient available resources to achieve high throughput and deliver accurate results. However, static allocation of stream processing resources in terms of machines is inefficient when input streams have significant rate variations— machines remain underutilised for long periods of average load. We present a combined stream processing system that, as the input stream rate varies, adaptively balances workload between a dedicated local stream processor and a cloud stream processor. This approach only utilises cloud machines when the local stream processor becomes overloaded. We evaluate a prototype system with financial trading data. Our results show that it can adapt effectively to workload variations, while only discarding a small percentage of input data.
http://lsds.doc.ic.ac.uk/research/doc/smdb11-cloud_streaming.pdf
Stream processing systems must handle stream data coming from real-time, high-throughput applications, for example in financial trading. Timely processing of streams is important and requires sufficient available resources to achieve high throughput and deliver accurate results. However, static allocation of stream processing resources in terms of machines is inefficient when input streams have significant rate variations— machines remain underutilised for long periods of average load. We present a combined stream processing system that, as the input stream rate varies, adaptively balances workload between a dedicated local stream processor and a cloud stream processor. This approach only utilises cloud machines when the local stream processor becomes overloaded. We evaluate a prototype system with financial trading data. Our results show that it can adapt effectively to workload variations, while only discarding a small percentage of input data.
2011年3月9日水曜日
[StreamGraph] 情報処理学会ACS論文誌・条件付き採録
雁瀬君が出したACS論文誌が条件付き採録。すべての条件を満たす必要が当然あるが、論文のわかりやすさを明確にするのはもちろんのこと、グラフ分割の再計算、マイグレーションの問題を解く必要がある。
[StreamGPU] 論文の方向性
上野君と議論し、今後2本の論文を出すことにする。
- GPUタスク並列の論文ーSVD と IKA-SST によって評価。IKA-SSTに関しては、カーネル実行とデータ転送のオーバーラップ、複数カーネル実行、データ差分転送による最適化は無しで純粋にタスク並列で勝負
- もう一本は、「異常検知アルゴリズム IKA-SST のGPUによる最適化」これはタスク並列＋カーネル実行とデータ転送のオーバーラップ、複数カーネル実行、データ差分転送による最適化あり
- GPUタスク並列の論文ーSVD と IKA-SST によって評価。IKA-SSTに関しては、カーネル実行とデータ転送のオーバーラップ、複数カーネル実行、データ差分転送による最適化は無しで純粋にタスク並列で勝負
- もう一本は、「異常検知アルゴリズム IKA-SST のGPUによる最適化」これはタスク並列＋カーネル実行とデータ転送のオーバーラップ、複数カーネル実行、データ差分転送による最適化あり
2011年3月8日火曜日
A list of international conferences / workshops
[2Q]
- IFIP Performance, 2011/04/23 締め切り
Submission deadline: April 22, 2011
Acceptance notification: June 30, 2011
- Middleware 2011, 2011/5/23 Abstract, 5/30 Paper Submission, 8/5 Notification
The Middleware 2011 proceedings will be published in Lecture Notes in Computer Science (LNCS) both in print and in electronic form. Papers must not exceed 20 pages, including abstract, all figures, tables, and references.
DISC: International Symposium on Distributed Computing
Registration & abstract submission: April 27th, 2011
Submission: May 1st, 2011
Acceptance notification: June 23rd, 2011
SOCC: ACM Symposium on Cloud Computing
Submission Deadline: April 30, 2011
Acceptance Notification: July 11, 2011
ICSOC
Abstract due : May 27, 2011
Full paper due : June 3, 2011
Notification : August 5, 2011
IMC: Internet Measurement Conference
May 6, 2011, 11:59PM EDT: Registration of title and 250-word abstract
May 13, 2011, 11:59PM EDT: Hard submission deadline
July 22, 2011: Notification
LADIS: ACM SIGOPS & SIGACT Workshop on Large Area Distributed Systems & Middleware
Submissions Due June 12, 2011
Notification of Acceptance :July 15, 2011
ASPLOS
Abstract deadline: Monday, July 18, 2011 (noon, EDT)
Full paper deadline: Monday, July 25, 2011 (noon, EDT)
Rebuttal period: Tuesday–Thursday, October 4–6, 2011
Notification of acceptance: Friday, October 28, 2011
IISWC (2011 IEEE International Symposium on Workload Characterization)
Abstracts Due: June 10, 2011
Paper Submission: June 17, 2011
Acceptance Notification: August 31, 2011
- ICDE 7月
Abstracts due July 12th, 2011
Full papers due July 19th, 2011
Author feedback on September 8th-13th, 2011
Author notification on September 27th, 2011
Full papers due July 19th, 2011
Author feedback on September 8th-13th, 2011
Author notification on September 27th, 2011
- CIKM: ACS Internal Conference on Information and Knowledge Management http://www.cikm2011.org/
- Abstracts due: 17th May, 2011
- Papers due: 24th May, 2011
- Notification of Acceptance: 19th July, 2011
[3Q]
CGO: Symposium on Code Generation and Optimization
Abstract Submission: September 13, 2011
Paper Submission: September 20, 2011
Author Rebuttal Period: October 23 to 26, 2011
Notification to Authors: November 1, 2011
EDBT
IEEE INFOCOM
IPDPS
SC
[1Q]
HotCloud:USENIX Workshop on Hot Topics in Cloud Computing
- Paper submissions due: March 14, 2011, 11:59 p.m. EDT Deadline Extended!
- Notification of acceptance: April 11, 2011
[その他」
KDD
ICDM
Mobicom
Mobisys
PACT
PLDI
SDM
SIGMOD/PODS
SOSP
Supercomputing
USENIX:
USENIX NSDI
VLDB
SRDS: IEEE International Symposium on Reliable Distributed Systems
PerCom: IEEE International Conference on Pervasive Computing and Communications
MobiQuitous: Mobile and Ubiquitous Systems
HotMobile: Workshop on Mobile Computing Systems and Applications
MDM: International Conference on Mobile Data Management
MobiHoc: ACM International Symposium on Mobile Ad Hoc Networks and Computing
SPAA: ACM Symposium on Parallelism in Algorithms and Architectures
CF: Computing Frontiers
VEE: Virtual Execution Environment
[秋]
- WWW, 12月
- ICDCS 11月後半
- ISPASS 11月20日
- ACM SIGMETRICS 11月初旬
MASCOTS
[1Q]
ACM SIGCOMM 1月
ICS: International Conference on Supercomputing, 1月締め切り
DEBS 2011
Abstracts for Research and Industry papers February 28, 2011 (23.59 GMT)
March 7, 2011 (23.59 GMT)
Tutorials, Research and Industry papers due March 7, 2011 (23.59 GMT)
March 14, 2011 (23.59 GMT)
Author notification for Tutorials, Research,
Industry and Ph.D. Workshop papers April 25, 2011
Abstract submission: 1 April 2011 (strict)
Paper and tool submission: 8 April 2011,
ICWS 1月中旬締め切り
Euro-Par: Parallel and Distributed Computing 1月締め切り
2011年3月5日土曜日
DEBS - ACM Conference on Distributed Event Based System
DEBS 2011 http://debs2011.fzi.de/
過去の DEBS のストリーム関連の論文。
DEBS 2010 http://debs10.doc.ic.ac.uk/
- Scalable, Elastic Distributed Stream Processing, David Alves
- Semantic Quality-Assurance in Distributed and Heterogeneous Stream Processing Systems
- Processing out-of-order event streams in ETALIS
- StreamNetFlux: Birth of Transparent Integrated CEP-DBs (demo)
- Flood: Elastic Streaming MapReduce (demo)
- Placement of Replicated Tasks for Distributed Stream Processing Systems (IBM Watson)
- Workload Characterization for Operator-Based Distributed Stream Processing Applications (IBM Watson)
- Evaluation of Streaming Aggregation on Parallel Hardware Architectures
DEBS 2009 http://debs09.isis.vanderbilt.edu/
- Distributed Event Stream Processing with Non-deterministic Finite Automata
- Implementing Reliable Event Streams in Large Systems via Distributed Data Flows and Recursive Delegation
- Processing Publish/Subscribe Queries over Distributed Data Streams (
DEBS 2008 http://debs08.dis.uniroma1.it/
- Replica Placement for High Availability in Distributed Stream Processing Systems
- Real-Time, Load-Adaptive Processing of Continuous Queries Over Data Streams
DEBS 2007 http://www.debs.msrg.utoronto.ca/
- Mythbusters: event stream processing versus complex event processing, Invited Talk, Tim Bass
- High frequency distributed data stream event correlation to improve neonatal clinical management
- A practical approach for enabling online analysis of event streams
- Persisting and querying biometric event streams with hybrid relational-XML DBMS
過去の DEBS のストリーム関連の論文。
DEBS 2010 http://debs10.doc.ic.ac.uk/
- Scalable, Elastic Distributed Stream Processing, David Alves
- Semantic Quality-Assurance in Distributed and Heterogeneous Stream Processing Systems
- Processing out-of-order event streams in ETALIS
- StreamNetFlux: Birth of Transparent Integrated CEP-DBs (demo)
- Flood: Elastic Streaming MapReduce (demo)
- Placement of Replicated Tasks for Distributed Stream Processing Systems (IBM Watson)
- Workload Characterization for Operator-Based Distributed Stream Processing Applications (IBM Watson)
- Evaluation of Streaming Aggregation on Parallel Hardware Architectures
DEBS 2009 http://debs09.isis.vanderbilt.edu/
- Distributed Event Stream Processing with Non-deterministic Finite Automata
- Implementing Reliable Event Streams in Large Systems via Distributed Data Flows and Recursive Delegation
- Processing Publish/Subscribe Queries over Distributed Data Streams (
DEBS 2008 http://debs08.dis.uniroma1.it/
- Replica Placement for High Availability in Distributed Stream Processing Systems
- Real-Time, Load-Adaptive Processing of Continuous Queries Over Data Streams
DEBS 2007 http://www.debs.msrg.utoronto.ca/
- Mythbusters: event stream processing versus complex event processing, Invited Talk, Tim Bass
- High frequency distributed data stream event correlation to improve neonatal clinical management
- A practical approach for enabling online analysis of event streams
- Persisting and querying biometric event streams with hybrid relational-XML DBMS
2011年3月3日木曜日
良い論文を書く
良い論文を書く（＝国際会議に通す）上で必要なことを書きます。
(1) 実装を始める前に、徹底的に既存研究を調査し、それとの差異を明確にする。差異が明確にできたら、深い実装を始める前にスライドに一つの筋書きが書けるはずです。研究によっては、少し実装を始めなければわからないこともあるでしょう。この際にはできるだけ、クイックにプロトタイプ実装し、iterative に(1) を繰り返していきましょう。
(2) 実装中には、新たな技術的発見、既存研究との差異をより明確化するような要素を見つけてください。その研究の価値が集りますし、新規性を主張しやすくなります。
(3) 実装が終了した後は、徹底的な定性的・定量的評価をしてください。(a) 評価対象アプリケーション・シナリオの豊富さ（マイクロベンチマーク、実アプリケーションを含めて3つが定石）、(b) 提案手法の優位性を示すデータ、(c) (b)に関するプロファイリング（なぜ良くなったか、なぜ悪いかを定量的に示す）(d) 類似技術との比較、の4点セットが必須です。特にほっとして(3)が poor になりがちですが、非常に大切です。
論文は最終的に国際会議（査読付き）に通してこそ、その研究が終了したと思ってください。(1) の段階で研究の方向性を変えていくことは悪いことではありません。むしろ研究の方向性が間違ったまま突っ走ることこそ良くありません。但し、(2) まで進んだ際には必ずその研究をしっかりと完遂していきましょう。
関連エントリ
(1) 実装を始める前に、徹底的に既存研究を調査し、それとの差異を明確にする。差異が明確にできたら、深い実装を始める前にスライドに一つの筋書きが書けるはずです。研究によっては、少し実装を始めなければわからないこともあるでしょう。この際にはできるだけ、クイックにプロトタイプ実装し、iterative に(1) を繰り返していきましょう。
(2) 実装中には、新たな技術的発見、既存研究との差異をより明確化するような要素を見つけてください。その研究の価値が集りますし、新規性を主張しやすくなります。
(3) 実装が終了した後は、徹底的な定性的・定量的評価をしてください。(a) 評価対象アプリケーション・シナリオの豊富さ（マイクロベンチマーク、実アプリケーションを含めて3つが定石）、(b) 提案手法の優位性を示すデータ、(c) (b)に関するプロファイリング（なぜ良くなったか、なぜ悪いかを定量的に示す）(d) 類似技術との比較、の4点セットが必須です。特にほっとして(3)が poor になりがちですが、非常に大切です。
論文は最終的に国際会議（査読付き）に通してこそ、その研究が終了したと思ってください。(1) の段階で研究の方向性を変えていくことは悪いことではありません。むしろ研究の方向性が間違ったまま突っ走ることこそ良くありません。但し、(2) まで進んだ際には必ずその研究をしっかりと完遂していきましょう。
関連エントリ
[StreamCloud] 情報処理学会全国大会発表終了
石井君の情報処理学会全国大会＠東工大の発表が終了。お疲れ様でした。
以下、指摘された点などを含めて、今後やるべきことを列挙
- 負荷予測アルゴリズム（SDAR)の向上とそれと組み合わせた実験を行う。負荷予測アルゴリズムの説明は必須。松浦君の StreamDS に関するロードバランシングに関する論文を読む。
- データストリーム処理以外の研究分野における類似技術の列挙とそれらとの差異
- Timeslot の概念と、 Amazon EC2 が現在定めている1時間単位の課金、の乖離の説明及び対応
- どのようなリアルタイム性が必要なアプリケーションかを定量的に明確にする
- 評価アプリケーションの種類を増やす。実験の大規模化。
[StreamDS] 負荷予測手法に関する論文
以下は負荷予測手法に関する関連論文3本。
Trace-based evaluation of job runtime and queue wait time predictions in grids
http://portal.acm.org/citation.cfm?id=1551609.1551632
Large-scale distributed computing systems such as grids are serving a growing number of scientists. These environments bring about not only the advantages of an economy of scale, but also the challenges of resource and workload heterogeneity. A consequence of these two forms of heterogeneity is that job runtimes and queue wait times are highly variable, which generally reduces system performance and makes grids difficult to use by the common scientist. Predicting job runtimes and queue wait times have been widely studied for parallel environments. However, there is no detailed investigation on how the proposed prediction methods perform in grids, whose resource structure and workload characteristics are very different from those in parallel systems. In this paper, we assess the performance and benefit of predicting job runtimes and queue wait times in grids based on traces gathered from various research and production grid environments. First, we evaluate the performance of simple yet widely used time series prediction methods and the effect of applying them to different types of job classes (e.g., all jobs submitted by single users or to single sites). Then, we investigate the performance of two kinds of queue wait time prediction methods for grids. Last, we investigate whether prediction-based grid-level scheduling policies can have better performance than policies that do not use predictions.
Swift: Fast, Reliable, Loosely Coupled Parallel Computation
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.92.519
A common pattern in scientific computing involves the execution of many tasks that are coupled only in the sense that the output of one may be passed as input to one or more others—for example, as a file, or via a Web Services invocation. While such “loosely coupled” computations can involve large amounts of computation and communication, the concerns of the programmer tend to be different than in traditional high performance computing, being focused on management issues relating to the large numbers of datasets and tasks (and often, the complexities inherent in “messy ” data organizations) rather than the optimization of interprocessor communication. To address these concerns, we have developed Swift, a system that combines a novel scripting language called SwiftScript with a powerful runtime system based on CoG Karajan and Falkon to allow for the concise specification, and reliable and efficient execution, of large loosely coupled computations. Swift adopts and adapts ideas first explored in the GriPhyN virtual data system, improving on that system in many regards. We describe the SwiftScript language and its use of XDTM to describe the logical structure of complex file system structures. We also present the Swift system and its use of CoG Karajan, Falkon, and Globus services to dispatch and manage the execution of many tasks in different execution environments. We summarize application experiences and detail performance experiments that quantify the cost of Swift operations. 1.
Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling
http://portal.acm.org/citation.cfm?id=1755940
As organizations start to use data-intensive cluster computing systems like Hadoop and Dryad for more applications, there is a growing need to share clusters between users. However, there is a conflict between fairness in scheduling and data locality (placing tasks on nodes that contain their input data). We illustrate this problem through our experience designing a fair scheduler for a 600-node Hadoop cluster at Facebook. To address the conflict between locality and fairness, we propose a simple algorithm called delay scheduling: when the job that should be scheduled next according to fairness cannot launch a local task, it waits for a small amount of time, letting other jobs launch tasks instead. We find that delay scheduling achieves nearly optimal data locality in a variety of workloads and can increase throughput by up to 2x while preserving fairness. In addition, the simplicity of delay scheduling makes it applicable under a wide variety of scheduling policies beyond fair sharing.
Generating Adaptation Policies for Multi-tier Applications in Consolidated Server Environments
http://portal.acm.org/citation.cfm?id=1444674
Creating good adaptation policies is critical to building complex autonomic systems since it is such policies that define the system configuration used in any given situation. While online approaches based on control theory and rule-based expert systems are possible solutions, each has its disadvantages. Here, a hybrid approach is described that uses modeling and optimization offline to generate suitable configurations, which are then encoded as policies that are used at runtime. The approach is demonstrated on the problem of providing dynamic management in virtualized consolidated server environments that host multiple multi-tier applications. Contributions include layered queuing models for Xen-based virtual machine environments, a novel optimization technique that uses a combination of bin packing and gradient search, and experimental results that show that automatic offline policy generation is viable and can be accurate even with modest computational effort.
Trace-based evaluation of job runtime and queue wait time predictions in grids
http://portal.acm.org/citation.cfm?id=1551609.1551632
Large-scale distributed computing systems such as grids are serving a growing number of scientists. These environments bring about not only the advantages of an economy of scale, but also the challenges of resource and workload heterogeneity. A consequence of these two forms of heterogeneity is that job runtimes and queue wait times are highly variable, which generally reduces system performance and makes grids difficult to use by the common scientist. Predicting job runtimes and queue wait times have been widely studied for parallel environments. However, there is no detailed investigation on how the proposed prediction methods perform in grids, whose resource structure and workload characteristics are very different from those in parallel systems. In this paper, we assess the performance and benefit of predicting job runtimes and queue wait times in grids based on traces gathered from various research and production grid environments. First, we evaluate the performance of simple yet widely used time series prediction methods and the effect of applying them to different types of job classes (e.g., all jobs submitted by single users or to single sites). Then, we investigate the performance of two kinds of queue wait time prediction methods for grids. Last, we investigate whether prediction-based grid-level scheduling policies can have better performance than policies that do not use predictions.
Swift: Fast, Reliable, Loosely Coupled Parallel Computation
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.92.519
A common pattern in scientific computing involves the execution of many tasks that are coupled only in the sense that the output of one may be passed as input to one or more others—for example, as a file, or via a Web Services invocation. While such “loosely coupled” computations can involve large amounts of computation and communication, the concerns of the programmer tend to be different than in traditional high performance computing, being focused on management issues relating to the large numbers of datasets and tasks (and often, the complexities inherent in “messy ” data organizations) rather than the optimization of interprocessor communication. To address these concerns, we have developed Swift, a system that combines a novel scripting language called SwiftScript with a powerful runtime system based on CoG Karajan and Falkon to allow for the concise specification, and reliable and efficient execution, of large loosely coupled computations. Swift adopts and adapts ideas first explored in the GriPhyN virtual data system, improving on that system in many regards. We describe the SwiftScript language and its use of XDTM to describe the logical structure of complex file system structures. We also present the Swift system and its use of CoG Karajan, Falkon, and Globus services to dispatch and manage the execution of many tasks in different execution environments. We summarize application experiences and detail performance experiments that quantify the cost of Swift operations. 1.
Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling
http://portal.acm.org/citation.cfm?id=1755940
As organizations start to use data-intensive cluster computing systems like Hadoop and Dryad for more applications, there is a growing need to share clusters between users. However, there is a conflict between fairness in scheduling and data locality (placing tasks on nodes that contain their input data). We illustrate this problem through our experience designing a fair scheduler for a 600-node Hadoop cluster at Facebook. To address the conflict between locality and fairness, we propose a simple algorithm called delay scheduling: when the job that should be scheduled next according to fairness cannot launch a local task, it waits for a small amount of time, letting other jobs launch tasks instead. We find that delay scheduling achieves nearly optimal data locality in a variety of workloads and can increase throughput by up to 2x while preserving fairness. In addition, the simplicity of delay scheduling makes it applicable under a wide variety of scheduling policies beyond fair sharing.
Generating Adaptation Policies for Multi-tier Applications in Consolidated Server Environments
http://portal.acm.org/citation.cfm?id=1444674
Creating good adaptation policies is critical to building complex autonomic systems since it is such policies that define the system configuration used in any given situation. While online approaches based on control theory and rule-based expert systems are possible solutions, each has its disadvantages. Here, a hybrid approach is described that uses modeling and optimization offline to generate suitable configurations, which are then encoded as policies that are used at runtime. The approach is demonstrated on the problem of providing dynamic management in virtualized consolidated server environments that host multiple multi-tier applications. Contributions include layered queuing models for Xen-based virtual machine environments, a novel optimization technique that uses a combination of bin packing and gradient search, and experimental results that show that automatic offline policy generation is viable and can be accurate even with modest computational effort.
登録:
投稿 (Atom)
