European Experts Exchange (EEE.org) :: Free Experts PHP MySql Oracle Microsoft Web SQL Delphi


Languages :: PHP :: Source Code of specified URL into a variable
By: blacksix	Date: 15/04/2003 00:00:00	Points: 400	Status: Answered Quality : Excellent
I'm looking to grab the source code from a website and put it into a variable so I can then go through and pick up a certain chunk of data and format it to fit my site format.... For example say i want to grab the top 10 web searches from a weekly report on lycose.com... i say something like $top10html = GETURL("<A HREF="http://lycose.com/blablabla/top10/index.html">http://lycose.com/blablabla/top10/index.html</a>"); and then work with the $top10html variable from there.... I've got no idea where to start... I've heard of CURL and fsockopen (i think) but dont really get them.. I just need a simple explaination lol Thanks!!
By: cirtap	Date: 15/04/2003 11:04:00		Type : Comment
Hi blacksix, do you think it's a good idea to "steal" someone else's content without permission? CirTap

By: blacksix	Date: 15/04/2003 12:02:00		Type : Comment
No I do not think that is a good or moral idea. I'm not doing this do "Steal" someone elses work... I design websites for an Irish record company and would like to post the schedule information from websites of the artist websites on our record labels. We have permission. We want to make one centralized website with the info, and the artists think its a good idea as well. The lycose thing was an example, and not my intention whatsoever. Sure I could like their sites, but my boss said he wants to keep it all in the same format, and what he asks for, it is my responsibility to produce. There may be other ways to do the same thing involving a centralized database of some sort but thats just too hard to set up across the different sites. Sorry, I should have clarified this.

By: blacksix	Date: 15/04/2003 15:51:00		Type : Comment
NEVERMIND I figured out how to do it using fopen() and fread()

By: lupuleine	Date: 16/04/2003 05:23:00		Type : Comment
hi so here's what you could do $lines = @file("<A HREF="http://lycose.com/blablabla/top10/index.html">http://lycose.com/blablabla/top10/index.html</a>"); then with an loop like for($i=0;$i<sizeof($lines);$i++) { //in $lines[$i] you have every single source code line } or you can use while loop and you check each line for the information you want it's ok to look for special tags that are around the information you look for Ps. Cirtap if really don't wanna help people around this forum, then please don't post any crap. It would be appreciated. 10x

By: VGR	Date: 16/04/2003 05:31:00		Type : Comment
agree with both, in fact 8-) Even if I'm a "stealer" ;-) It's not the source code of the site that gets retrieved (and not "stolen") but the HTML generated by it ; this is publicly displayed, after all, and any browser can read it, so why not a PHP script ? I think you figured it out ; fine. All you've left is to parse the results for the data you search. I may help there too, althought thisis a very common question whose answer is in the recent PAQ ;-)

By: TheFalklands	Date: 16/04/2003 19:04:00		Type : Comment
function GetUrl($url) { $lines = file($url); for($i=0;$i<sizeof($lines);$i++) { $all_lines .= $lines[$i]; } return $all_lines; } $url = "<A HREF="http://lycose.com/blablabla/top10/index.html">http://lycose.com/blablabla/top10/index.html</a>"; $top10html = GetUrl($url);

By: VGR	Date: 16/04/2003 19:14:00		Type : Comment
yes, yes, yes everybody knows how to do that :-) the difficult part is to parse efficiently (and clearly, for you regexp guys around there :D ) the $all_lines[]

By: bljak	Date: 16/04/2003 21:57:00		Type : Comment
tsk tsk not good idea saying CirTap doesn't help, if you have had took a look at his profile before saying that you would know better, however, has nothing to do with this question, i just gone mad p.s. god bless regexps :P //bljak

By: lupuleine	Date: 16/04/2003 22:38:00		Type : Comment
I looked at cirtap's profile and I respect his/her knowledge but, the given answer was not for help to anyone on this forum. Sorry, I only say what I have to say. ;)

By: blacksix	Date: 16/04/2003 23:19:00		Type : Comment
so I got it working before i got an answer and i cant delete the question so im not just gonna toss those points... althought I appreciate the effort, I figure i'll ask another question here if its not agains some kind of rule... This is a very slow script.. so what i would like to do is write a script that grabs the html chunk every couple hours (without being manually accessed) and stores it locally for much faster retrieval. How would I start this? I know this kind of thing can be done with databases so I figure it can be done here too... THANKS GUYS!!

By: VGR	Date: 16/04/2003 23:25:00		Type : Comment
easy. You can program the retrieval to take place every 2 hours using my free RobotOuaibe utility, or (if PHP is cgi exec) using AT or the crontab, depending on your platform. You can also (solution of the poor :D ) leave open a browser window with a refresh time of 7200 seconds ;-) interested ?

By: VGR	Date: 16/04/2003 23:26:00		Type : Comment
as for your question, can't you reduce thepoints yourself ? Or ask for a refund in feedback area ?

By: blacksix	Date: 16/04/2003 23:45:00		Type : Comment
thanks VGR its UNIX so thats crontab right? can you tell me a little about it? The server admin has to set that up correct? The points are yours just point me in the right direction. Thanks once again

By: bljak	Date: 16/04/2003 23:54:00		Type : Comment
no need for your admin to set it up you can do that yourself man crontab and then use it //bljak

By: VGR	Date: 17/04/2003 00:02:00		Type : Answer
yes the crontab is usually manipulated, if my souvenirs are correct, with command : crontab (it konws which user you are and the entries you'lla dd will be yours only) Note that this needsto be applied using commands like : php nameofscript.php

By: blacksix	Date: 17/04/2003 00:06:00		Type : Comment
VGR is da man

By: bljak	Date: 17/04/2003 00:10:00		Type : Comment
is a man? damn, i already tried to flirt with him, thinking its woman :P //bljak

By: VGR	Date: 17/04/2003 00:28:00		Type : Comment
huh huh huh where do you come from bljak?

Do register to be able to answer

Add This Article To:
Del.icio.us	Digg	Google	Spurl
Blink	Wong	Simpy	Y! MyWeb

Contact

page generated in 72.648050 milliseconds

Why Google AdSense ads ?

New Improved Search!