• We've upgraded our forums. Please post any issues/requests in this thread.

Parse Web Page to MySQL Db

Joined
Aug 17, 2008
Messages
2,184 (0.64/day)
Likes
1,027
Location
Minnesota, USA
System Name TIGRAlpha4
Processor i5 750 @ 3.6GHz
Motherboard ASRock P55 Deluxe
Cooling S1283 + FFB1212VHE
Memory 16GB DDR3 1800, 7-8-7-24
Video Card(s) 560 Ti
Storage 128GB Kingston SSD + 16TB RAID 6
Display(s) Dell P1130 + Samsung 275T
Case Antec Twelve Hundred
Power Supply Silverstone OP1000-E
#1
accidental edit
 
Last edited:

W1zzard

Administrator
Staff member
Joined
May 14, 2004
Messages
17,059 (3.44/day)
Likes
17,963
Processor Core i7-4790K
Memory 16 GB
Video Card(s) GTX 1080
Display(s) 30" 2560x1600 + 19" 1280x1024
Software Windows 7
#2
just fetch the page via curl functions or file_get_contents, extract whatever info you need via simple string processing or preg_match and put it in your db
 
Joined
Aug 17, 2008
Messages
2,184 (0.64/day)
Likes
1,027
Location
Minnesota, USA
System Name TIGRAlpha4
Processor i5 750 @ 3.6GHz
Motherboard ASRock P55 Deluxe
Cooling S1283 + FFB1212VHE
Memory 16GB DDR3 1800, 7-8-7-24
Video Card(s) 560 Ti
Storage 128GB Kingston SSD + 16TB RAID 6
Display(s) Dell P1130 + Samsung 275T
Case Antec Twelve Hundred
Power Supply Silverstone OP1000-E
#3
just fetch the page via curl functions or file_get_contents, extract whatever info you need via simple string processing or preg_match and put it in your db
Someone who properly knew what he was doing would probably understand that really well. As for me, I have some searching and studying to do.

A recent head injury has left me hazy on much of the coding and programming I once understood. At this point, all I understand is basic PHP and basic use of phpMyAdmin for MySQL. I am learning all over, as I go.

Thank you for getting me started.
 
Joined
Aug 10, 2007
Messages
4,059 (1.07/day)
Likes
1,123
Location
Geneva, FL, USA
Processor Intel i5-6600
Motherboard ASRock H170M-ITX
Cooling Cooler Master Geminii S524
Memory G.Skill DDR4-2133 16GB (8GB x 2)
Video Card(s) Gigabyte R9-380X 4GB
Storage Samsung 950 EVO 250GB (mSATA)
Display(s) LG 29UM69G-B 2560x1080 IPS
Case Lian Li PC-Q25
Audio Device(s) Realtek ALC892
Power Supply Seasonic SS-460FL2
Mouse Logitech G700s
Keyboard Logitech G110
Software Windows 10 Pro
#4
The first table.

PHP:
$page = 'http://fah-web.stanford.edu/cgi-bin/main.py?teamnum=174132&qtype=teampage';
$html = file_get_contents($page);
$html = str_replace(array("\t", "\n"), '', $html);

$pattern = '~<TD align=left>\s(.*)\s*(</TD>|\()~U';
preg_match_all($pattern, $html, $matches);
$matches[1] holds an array with the following:

PHP:
[0] => 06:13:59 November 02, 2010
[1] => 2010-11-02 06:12:46
[2] => 35
[3] => 174132
[4] => 10016245
[5] => 22377
[6] => 556 of 189735
[7] => <a href=http://www.re-hq.net> http://www.re-hq.net </a>
That handles a few of the items you want. Team ranking can come from:

PHP:
list($rank, $teamCount) = explode(' of ', $matches[1][6]);
Or if team count isn't important, could just modify the pattern a little: '~<TD align=left>\s(.*)\s*(</TD>|\(| of)~U'. Then $matches[1][6] is just the team rank.
 
Last edited:
Joined
Aug 17, 2008
Messages
2,184 (0.64/day)
Likes
1,027
Location
Minnesota, USA
System Name TIGRAlpha4
Processor i5 750 @ 3.6GHz
Motherboard ASRock P55 Deluxe
Cooling S1283 + FFB1212VHE
Memory 16GB DDR3 1800, 7-8-7-24
Video Card(s) 560 Ti
Storage 128GB Kingston SSD + 16TB RAID 6
Display(s) Dell P1130 + Samsung 275T
Case Antec Twelve Hundred
Power Supply Silverstone OP1000-E
#5
The first table.

PHP:
$page = 'http://fah-web.stanford.edu/cgi-bin/main.py?teamnum=174132&qtype=teampage';
$html = file_get_contents($page);
$html = str_replace(array("\t", "\n"), '', $html);

$pattern = '~<TD align=left>\s(.*)\s*(</TD>|\()~U';
preg_match_all($pattern, $html, $matches);
$matches[1] holds an array with the following:

PHP:
[0] => 06:13:59 November 02, 2010
[1] => 2010-11-02 06:12:46
[2] => 35
[3] => 174132
[4] => 10016245
[5] => 22377
[6] => 556 of 189735
[7] => <a href=http://www.re-hq.net> http://www.re-hq.net </a>
That handles a few of the items you want. Team ranking can come from:

PHP:
list($rank, $teamCount) = explode(' of ', $matches[1][6]);
Sir, I am much in your debt. This is invaluable to me. I will be reviewing your code and working on this project today. Thank you.
 
Joined
Aug 10, 2007
Messages
4,059 (1.07/day)
Likes
1,123
Location
Geneva, FL, USA
Processor Intel i5-6600
Motherboard ASRock H170M-ITX
Cooling Cooler Master Geminii S524
Memory G.Skill DDR4-2133 16GB (8GB x 2)
Video Card(s) Gigabyte R9-380X 4GB
Storage Samsung 950 EVO 250GB (mSATA)
Display(s) LG 29UM69G-B 2560x1080 IPS
Case Lian Li PC-Q25
Audio Device(s) Realtek ALC892
Power Supply Seasonic SS-460FL2
Mouse Logitech G700s
Keyboard Logitech G110
Software Windows 10 Pro
#6
No prob. My regex isn't always the best or most efficient pattern (I don't do enough of it to get the experience), but if you have some troubles, I can probably help on the second table tonight.
 

Helli

New Member
Joined
Feb 15, 2010
Messages
57 (0.02/day)
Likes
12
Location
Germany, near Munich
Processor Intel Xeon E5620
Motherboard Asus Rampage III Extreme
Cooling Thermalright Silver Arrow
Memory 3x2GB A-Data DDR3-2000 CL9
Video Card(s) 2x GTX 590 (MSI & PNY)
Storage Samsung HD204UI/HD501LJ
Display(s) 24" Samsung SyncMaster 245B
Case Aerocool Xpredator
Audio Device(s) Onboard
Power Supply Antec TruePower Quattro 1200W
Software Microsoft Windows 7 Professional x64 Edition
#7
I have done this on other Projects for my Team.
Keep in Mind, on every Code Change the Staff will do in the Future
you have to modify your Script also. That can mean much, much work. ;)

Helli