my-asu-archive/real-time-traffic-analysis-clickhouse
1
0
Fork 0
mirror of https://github.com/20kaushik02/real-time-traffic-analysis-clickhouse.git synced 2026-01-25 08:04:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

preprocessing data

Browse Source
This commit is contained in:
Kaushik Narayan R
2024-10-31 00:29:26 -07:00
parent a96c4320b0
commit 957202d86f
2 changed files with 56 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
preprocessing/README.md Normal file
View File

@@ -0,0 +1,15 @@
# Data filtering, preprocessing and selection for further use
- IP packet traces are taken [from here](https://mawi.wide.ad.jp/mawi/samplepoint-F/2023/), specifically from 2023/10/01-2023/10/31 (yet to confirm)
- Filtering - TODO
- L4 - Limit to TCP and UDP
- maybe GRE for VPN usage?
- L3 - IPv6 is only around 10%, let's drop it
- Selection (of fields):
- Timestamp - note: capture window is from 0500-0515 UTC
- IP
- addresses - src, dst
- protocol - 6 (TCP) or 17 (UDP). cld go for boolean to save space
- TCP
- ports - sport, dport
- Packet size - in bytes - could exclude L2?