Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Q
qg-dbc-spark
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
data-spider
qg-dbc-spark
Commits
69b9d63b
Commit
69b9d63b
authored
Dec 27, 2019
by
data爬虫-冯 军凯
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
transactionLog 数据清洗 main类修改12345678
parent
224949ac
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
CleanningTransactionLogMain.java
...dbc/spark/transactionlog/CleanningTransactionLogMain.java
+2
-2
No files found.
src/main/java/cn/quantgroup/dbc/spark/transactionlog/CleanningTransactionLogMain.java
View file @
69b9d63b
...
...
@@ -32,7 +32,7 @@ public class CleanningTransactionLogMain {
ArrayList
<
TransactionLog
>
transactionLogs
=
new
ArrayList
<>();
String
sql
=
"INSERT INTO `call_record` (`request_url`, `transaction_id`, `uuid`, `url_type`, `code`, `created_at`, `updated_at`) VALUES (?,?,?,?,?,?,?)"
;
func
.
forEachRemaining
(
item
->
{
System
.
out
.
println
(
"单条数据清洗"
);
//
System.out.println("单条数据清洗");
try
{
String
[]
split
=
item
.
split
(
"\t"
);
TransactionLog
transactionLog
=
new
TransactionLog
();
...
...
@@ -46,8 +46,8 @@ public class CleanningTransactionLogMain {
transactionLog
.
setUpdated_at
(
timestamp
);
if
(
transactionLogs
.
size
()
!=
0
&&
transactionLogs
.
size
()
%
200
==
0
)
{
JdbcExecuters
.
prepareBatchUpdateExecuteTransactionid
(
sql
,
transactionLogs
);
System
.
out
.
println
(
"执行sql集合: "
+
transactionLogs
.
size
());
JdbcExecuters
.
prepareBatchUpdateExecuteTransactionid
(
sql
,
transactionLogs
);
transactionLogs
.
clear
();
}
}
catch
(
Exception
e
)
{
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment