Showing posts with label smfGetYahooPortfolioView. Show all posts
Showing posts with label smfGetYahooPortfolioView. Show all posts

Thursday, June 8, 2023

Some detail on the YPV fix

 Some details on the fix for the smfGetYahooPortfolioView (YPV since I don't like typing that much!)

The Yahoo data was still there, but it was blocked unless you were on their site.  If you went to https://finance.yahoo.com/ and then went to MyPortfolio, you could select the data for any stocks you added to your portfolio.  The API that people have been using the last number of years for Yahoo is not a supported public API.  It is in place for Yahoo's web pages to fetch the data to display on their web pages.  However, some people have been using these unsupported APIs to get market data... and the SMF Add-In is one such example.  It will retrieve close to 90 data elements in relation to the data provided on the Portfolio View page.

 When Yahoo changed the API to require authentication, it was very disappointing (I did hear some stronger words used) to many people.  I did get almost all of those data elements working with the RCHGetElementNumber funtion, but the problem with that function was it made so many calls to Yahoo's pages to get the elements for each ticker.  That was the main reason to move people to use the YPV function... it could be array entered into Excel and then it would retrieve all the requested elements for all the requested tickers in ONE CALL to Yahoo and return all that data in JSON format.  Then the VBA module could parse all the data out and insert it into the Excel sheet.

One of the members, Bruce, found some info on retrieving the crumb info from yahoo and while I was working on that, Tony (another member of the group) put together the fix while I went to get some sleep... procrastination wins again!

So now smfGetYahooPortfolioView has an additional parameter internally for calling RCHGetWebData which really just calls smfGetWebPage.  That calls RCHGetURLData when then in the case of the new fix, calls a new function RCHGetURLData4.  That new function calls another new function called getYahooCookieCrumb that gets and stores the cookie and crumb data for the resulting calls.  This allows the add-in to fetch the data by making the web page think it has been authorized to access it as an online user would be.

NOTE:

Even though I added over 100 new elements to the RCHGetElementNumber function, only use it for when you need a very few items.  For spreadsheets of lots of data, use the YPV function as an ARRAY ENTERED FORMULA.

 If you want reliable real time data for many many tickers and plan to keep it up to date all day long, please read Yahoo's rules on data usage and look at getting a streaming data feed.  If people abuse the data from Yahoo, they will continue to make changes with absolutely no notice to curtail the unwelcome traffic!

SMF new Add-In version 3.0.2023.06.08

 The function smfGetYahooPortfolioView is now fixed!

Thanks to help from the group, they provided information and code changes to utilize cookie and crumb into the web calls so that portfolio view will now once again return the JSON data.

The code updates are on my GitHub and the new version is posted on the SMF website.

Monday, May 8, 2023

SMF new Add-In version 3.0.2023.05.08

I have posted an updated version 3.0.2023.05.08 to the website
The new version
I also documented my babblings while working on it for any coders that want to check it out.  It is on GitHub Issue #27

I fixed a few more broken links such as the "Install the Add-In" link and others.
I'm also putting together a document on how to update / create an Excel Add-in for any that want to update their own version of the XLA file.

The fix changes the v7 to v6 (turns out the Export button on Yahoo Finance Portfolio page also changed to use that).  I also changed it in smf-elements-22.txt as it is a Constant for Yahoo Portfolio View.

I'm thing that may be able to be incorporated into some of the other modules so if there are changes to Yahoo's base URLs, it could be changed in there instead of always creating a new XLA file... just an idea.  It would take some coding and testing to incorporate it, but I use external files for other programs all the time so it might help.

Cheers,

Friday, May 5, 2023

SMF issue with Yahoo Portfolio

 Many of the users in https://groups.google.com/g/smf-addin reported yesterday that there were issues with the smfGetYahooPortfolioView module.

 One user noted that if the v7 in the URL = "https://query1.finance.yahoo.com/v7/finance/quote" was changed to v6 it works.

The v7 is used in 14 occurrences throughout the code base and in the following modules (occurrences):

  • smfGetOptionExpirations (1)
  • smfGetOptionStrikes (3)
  • RCHGetYahooHistory2 (1)
  • smfGetYahooJSONField (1)
  • RCHGetYahooQuotes (2)
  • smfGetYahooHistory (1)
  • smfGetYahooHistoryCSV (2)
  • smfGetYahooPortfolioView (1)
  • RCHGetYahooQuotes-Function.htm (web documentation)
  • smf-elements-22.txt (elements file for module RCHGetElementNumber)

 I think RCHGetYahooHistory2 is no longer in use.  I should look at removing some of that deprecated code, but it will be a lot of work the way everything is so intertwined.

Once I check these modules to see if the change in URL should be done for all, I will create a new add-in version and upload it to the website: https://climbermel.github.io/SMF_Add-in/

Thursday, April 20, 2023

smfGetYahooPortfolioView working again

 Today it started working again.  I can also log in with my old yahoo account and password, so there was obviously an issue on Yahoo.  There was also a message about invalid crumb, so it could be related to the failed login and cookies.

Wednesday, April 19, 2023

smfGetYahooPortfolioView not returning data

 Yahoo Portfolio View has stopped working


smfGetYahooPortfolioView was returning no data.  I was unable to log into portfolio view at Yahoo and the error returned was "unauthorized" when trying to get the json data.

There is also a notice of Portfolio 2.0 coming...

 Checking the code and the URL being created is also nott working if manually entered into a browser.

Here are some of the URLs I was testing with:

https://query1.finance.yahoo.com/v7/finance/quote?fields=" & sFieldList & "&formatted=false&symbols=" & replace(sTickers, ",XXXXX", "")

https://query1.finance.yahoo.com/v7/finance/quote?fields=longName,shortName,quoteSourceName,regularMarketPrice,regularMarketTime,regularMarketChange,twoHundredDayAverage,fiftyTwoWeekRange&formatted=false&symbols=MMM,ED,MCD,NWN,SPHD,XOM,XOM160812C0008650"

"https://query1.finance.yahoo.com/v7/finance/quote?fields=longName,shortName&formatted=false&symbols=AAPL"

"https://query1.finance.yahoo.com/v10/finance/quoteSummary/MMM?modules=financialData"

https://query1.finance.yahoo.com/v10/finance/quoteSummary/AAPL?modules=financialData

https://query1.finance.yahoo.com/v7/finance/quote?fields=longName,shortName&formatted=false&symbols=AAPL

The v10 was returning json data.


Tuesday, February 21, 2023

New Release in Progress

 I have added these issues to the next release:

    - smfGetYahooPortfolioView possible incorrect data returned #9

    -  smfGetYahooHistory close / adjclose using wrong pItem codes #19

    -  Duplicate Excel 2010 Function #18

 

There was no issue with smfGetYahooPortfolioView.  The user was also calling smfGetYahooHistory and that is where they were getting unadjusted intead of adjusted close

Issue# 18 is some unused code in the add-in that matched an Excel internal function and will be deleted.  It may have been originally written before Excel added the IFERROR() function.

The following modules will be updated:

RCHGetYahooHistory() & smfGetYahooHistory to change the Items back to C for adjusted close and U for unadjusted close.  A will now be allowed as well, but simply be converted to use C for adjusted.

smfUtilities_ to remove function IfError().

modGetElementNumber to update version number


SMF new Add-In version 3.0.2024.04.29

  This new release has fixes for: module smfGetOptionQuotes A member (Bruce) found the issue was with the crumb processing and provided cod...