Wednesday, September 20, 2023

SMF new Add-In version 3.0.2023.09.20

This new release has added functionality for smfGetYahooJSONData to work with 64bit Excel.

Thanks again to the growing help from the group.  This add-in is definitely becoming a group effort.  There are a number of fixed elements that have been submitted and are in the new smf-elements2.txt and smf-elements9.txt

I have updated the web site https://climbermel.github.io/SMF_Add-in/

A word of caution! 

 If you don't use the function smfGetYahooJSONData or if you don't use 64bit Excel, you may not want to use this change.  You can still extract the element files to update those without changing you xla file.

It is always a good idea to back up you C:\SMF Add-In folder before updating.

For example: copy C:\SMF Add-In to C:\Old Add-In

That way if it doesn't work as planned you can just copy it back...

Cheers,

Mel

Sunday, June 18, 2023

Notice that Zack's had a data breach

Just in case you use Zack's web site:

This notice is regarding a prior data breach disclosed by Zacks at https://www.zacks.com/breach/. We have confirmed that in association with this breach relating to a smaller subset of customers whose unencrypted passwords were compromised, the unauthorized third parties also gained access to encrypted passwords of zacks.com customers, but only in the encrypted format.

Wednesday, June 14, 2023

New and updated documentation

 I have updated a number of the pages on the web site.

I added a page to array enter a formula

How to fix links when Excel hard codes the location

Added documentation for smfStrExtr function and smfWord function

Why does RCHGetElementNumber return Undefined or Error

I cleaned off all the broken links from Tips & FAQs page.  I will add to it as I get more documentation.

Thursday, June 8, 2023

Some detail on the YPV fix

 Some details on the fix for the smfGetYahooPortfolioView (YPV since I don't like typing that much!)

The Yahoo data was still there, but it was blocked unless you were on their site.  If you went to https://finance.yahoo.com/ and then went to MyPortfolio, you could select the data for any stocks you added to your portfolio.  The API that people have been using the last number of years for Yahoo is not a supported public API.  It is in place for Yahoo's web pages to fetch the data to display on their web pages.  However, some people have been using these unsupported APIs to get market data... and the SMF Add-In is one such example.  It will retrieve close to 90 data elements in relation to the data provided on the Portfolio View page.

 When Yahoo changed the API to require authentication, it was very disappointing (I did hear some stronger words used) to many people.  I did get almost all of those data elements working with the RCHGetElementNumber funtion, but the problem with that function was it made so many calls to Yahoo's pages to get the elements for each ticker.  That was the main reason to move people to use the YPV function... it could be array entered into Excel and then it would retrieve all the requested elements for all the requested tickers in ONE CALL to Yahoo and return all that data in JSON format.  Then the VBA module could parse all the data out and insert it into the Excel sheet.

One of the members, Bruce, found some info on retrieving the crumb info from yahoo and while I was working on that, Tony (another member of the group) put together the fix while I went to get some sleep... procrastination wins again!

So now smfGetYahooPortfolioView has an additional parameter internally for calling RCHGetWebData which really just calls smfGetWebPage.  That calls RCHGetURLData when then in the case of the new fix, calls a new function RCHGetURLData4.  That new function calls another new function called getYahooCookieCrumb that gets and stores the cookie and crumb data for the resulting calls.  This allows the add-in to fetch the data by making the web page think it has been authorized to access it as an online user would be.

NOTE:

Even though I added over 100 new elements to the RCHGetElementNumber function, only use it for when you need a very few items.  For spreadsheets of lots of data, use the YPV function as an ARRAY ENTERED FORMULA.

 If you want reliable real time data for many many tickers and plan to keep it up to date all day long, please read Yahoo's rules on data usage and look at getting a streaming data feed.  If people abuse the data from Yahoo, they will continue to make changes with absolutely no notice to curtail the unwelcome traffic!

SMF new Add-In version 3.0.2023.06.08

 The function smfGetYahooPortfolioView is now fixed!

Thanks to help from the group, they provided information and code changes to utilize cookie and crumb into the web calls so that portfolio view will now once again return the JSON data.

The code updates are on my GitHub and the new version is posted on the SMF website.

Thursday, May 25, 2023

Now smfGetYahooPortfolioView has broken

 I put in fixes for one function and Yahoo breaks another!

So smfGetYahooPortfolioView started out as being flaky... It would work then it would give an error, then work... 

Today I can't get anything using the API to work.  The json response for YahooPortfolio gives a 404 not found or unauthorized access depending on how you try to access it.

I found this developer feedbak form:

 
Yahoo Finance | API Feedback
We’re sorry for the inconvenience, but API-level access to Yahoo Finance quotes data has been disabled.

Yahoo Finance licenses data from 3rd-party providers that do not currently authorize us to redistribute these data in API form. Licenses that authorize redistribution come with a greater cost that varies depending on a number of factors, including whether the data is for personal or commercial use, the type of data, the volume of queries, and additional features which may be available.

We would appreciate your feedback to ensure that we can continue to serve your needs. By understanding your intended use of these API data, we will be better able to acquire the appropriate licenses. We appreciate your feedback, and we read every response.

What types of data are you interested in accessing via an API?
Please select all that apply.
What kind of applications have you developed that used API data?
Please select all that apply.
Is the application you developed for personal or commercial use?
Would you be willing to pay for API data access?
If yes, what price per month (US dollars) would you be willing to pay?

Again, we are sorry for the inconvenience, and we thank you for your feedback.  If you would like to help us craft this new API service, please provide your email address so that we may contact you.

Tuesday, May 23, 2023

ZACKS Elements fixed

 I have fixed a number of Zacks data elements.  I was able to fix 852 that was marked Obsolete and I also added some new elements for the Earnings ESP.

The ESP (Expected Surprise Prediction) only had the current ESP 13894.  I added 13897 - 13899 for the ESP previous Quarter, 2 Quarter previous and 3 Quarters previous

 

Here are the fixes added to smf-elements-20.txt for now:

844;Zacks;Style Scores, Value;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","span",-1,"rank_view","rank_view","composite_val")
845;Zacks;Style Scores, Growth;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","span",2,"rank_view","rank_view","composite_val")
846;Zacks;Style Scores, Momentum;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","span",4,"rank_view","rank_view","composite_val")
847;Zacks;Style Scores, VGM;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","span",6,"rank_view","rank_view","composite_val")
848;Zacks;Zacks Rank;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","p",-1,"rank_view",,,,,,,,"<span")
852;Zacks;Rank Within Industry;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","a",-1,"rank_view","rank_view","rank_view","stocks/industry-rank")
13869;Zacks;Market Capitalization;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","span",1,">Market Cap")
13871;Zacks;Current Year Estimate;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","dd",1,">Current Yr Est")
13872;Zacks;EPS Last Year;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","dd",1,">Prior Year EPS<")
13873;Zacks;Zacks Recommendation;="Obsolete -- No data found"
13891;Zacks;Beta;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","span",1,">Beta")
13893;Zacks;Next Earnings Date;=smfstrExtr(smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~","dd",-1,"spl_sup_text"),"","~")
13894;Zacks;Earnings Expected Surprise Prediction current;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~/detailed-estimates","span",1,">Earnings ESP Stocks")
13897;Zacks;Earnings Expected Surprise Prediction prev Qtr;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~/detailed-estimates","span",2,">Earnings ESP Stocks")
13898;Zacks;Earnings Expected Surprise Prediction 2Q prev;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~/detailed-estimates","span",3,">Earnings ESP Stocks")
13899;Zacks;Earnings Expected Surprise Prediction3Q prev;=smfGetTagContent("https://www.zacks.com/stock/quote/~~~~~/detailed-estimates","span",4,">Earnings ESP Stocks")

Saturday, May 20, 2023

SMF new Add-In version 3.0.2023.05.20

 

I have posted an updated version 3.0.2023.05.20 to the website
The new version fixes these errors.  New processing added to modGetTagContent. smfGetTagContent to handle element 848 and smf-elements-6.txt updated to fix element 13892.

The issues are detailed on GitHub
 
Issue #31    RCHGetElement #13892 is wrong 
Issue #32    RCHGetElement #848 is wrong

Monday, May 8, 2023

SMF new Add-In version 3.0.2023.05.08

I have posted an updated version 3.0.2023.05.08 to the website
The new version
I also documented my babblings while working on it for any coders that want to check it out.  It is on GitHub Issue #27

I fixed a few more broken links such as the "Install the Add-In" link and others.
I'm also putting together a document on how to update / create an Excel Add-in for any that want to update their own version of the XLA file.

The fix changes the v7 to v6 (turns out the Export button on Yahoo Finance Portfolio page also changed to use that).  I also changed it in smf-elements-22.txt as it is a Constant for Yahoo Portfolio View.

I'm thing that may be able to be incorporated into some of the other modules so if there are changes to Yahoo's base URLs, it could be changed in there instead of always creating a new XLA file... just an idea.  It would take some coding and testing to incorporate it, but I use external files for other programs all the time so it might help.

Cheers,

Friday, May 5, 2023

New version in progress

 

I'm testing all the affected modules and I hope to have a new version of the add-in put together later today.  I'll let you know as soon as I have it posted to the website.

For any interested in GitHub or the process of dealing with a code fix, here is the issue on GitHub.  I'll track the process and changes on there.

Mel

SMF issue with Yahoo Portfolio

 Many of the users in https://groups.google.com/g/smf-addin reported yesterday that there were issues with the smfGetYahooPortfolioView module.

 One user noted that if the v7 in the URL = "https://query1.finance.yahoo.com/v7/finance/quote" was changed to v6 it works.

The v7 is used in 14 occurrences throughout the code base and in the following modules (occurrences):

  • smfGetOptionExpirations (1)
  • smfGetOptionStrikes (3)
  • RCHGetYahooHistory2 (1)
  • smfGetYahooJSONField (1)
  • RCHGetYahooQuotes (2)
  • smfGetYahooHistory (1)
  • smfGetYahooHistoryCSV (2)
  • smfGetYahooPortfolioView (1)
  • RCHGetYahooQuotes-Function.htm (web documentation)
  • smf-elements-22.txt (elements file for module RCHGetElementNumber)

 I think RCHGetYahooHistory2 is no longer in use.  I should look at removing some of that deprecated code, but it will be a lot of work the way everything is so intertwined.

Once I check these modules to see if the change in URL should be done for all, I will create a new add-in version and upload it to the website: https://climbermel.github.io/SMF_Add-in/

Thursday, April 20, 2023

smfGetYahooPortfolioView working again

 Today it started working again.  I can also log in with my old yahoo account and password, so there was obviously an issue on Yahoo.  There was also a message about invalid crumb, so it could be related to the failed login and cookies.

Wednesday, April 19, 2023

smfGetYahooPortfolioView not returning data

 Yahoo Portfolio View has stopped working


smfGetYahooPortfolioView was returning no data.  I was unable to log into portfolio view at Yahoo and the error returned was "unauthorized" when trying to get the json data.

There is also a notice of Portfolio 2.0 coming...

 Checking the code and the URL being created is also nott working if manually entered into a browser.

Here are some of the URLs I was testing with:

https://query1.finance.yahoo.com/v7/finance/quote?fields=" & sFieldList & "&formatted=false&symbols=" & replace(sTickers, ",XXXXX", "")

https://query1.finance.yahoo.com/v7/finance/quote?fields=longName,shortName,quoteSourceName,regularMarketPrice,regularMarketTime,regularMarketChange,twoHundredDayAverage,fiftyTwoWeekRange&formatted=false&symbols=MMM,ED,MCD,NWN,SPHD,XOM,XOM160812C0008650"

"https://query1.finance.yahoo.com/v7/finance/quote?fields=longName,shortName&formatted=false&symbols=AAPL"

"https://query1.finance.yahoo.com/v10/finance/quoteSummary/MMM?modules=financialData"

https://query1.finance.yahoo.com/v10/finance/quoteSummary/AAPL?modules=financialData

https://query1.finance.yahoo.com/v7/finance/quote?fields=longName,shortName&formatted=false&symbols=AAPL

The v10 was returning json data.


Thursday, March 30, 2023

Using the smfUpdateDownloadTable macro

---In smf_addin@yahoogroups.com, <rharmelink@...> wrote :

See also [Tip 2.1a] below from Randy.

This will give some visuals of creating a range to fill using the smfUpdateDownloadTable macro:

  1. The upper left hand corner cell of the table needs to have a range name of "Ticker" (see green-shaded cell on diagram). The text in the cell can be anything you want it to be.
  2. The cells below the "Ticker" cell should be filled in with Yahoo ticker symbols, one per cell (see yellow-shaded cells on diagram).
  3. The cells to the right of the "Ticker" cell should be filled with column titles (see orange-shaded cells on diagram. The text in these cells can be anything you want them to be.
  4. The cells above the column titles need to be filled in with SMF add-in formulas or element numbers (see blue-shaded cells on diagram). Use five tildas as a substitute for a ticker symbol. For example, any of the following text strings could be used to get "Market Capitalization" from Yahoo:

    941
    RCHGetElementNumber("~~~~~", 941)
    RCHGetTableCell("http://finance.yahoo.com/q/ks?s=~~~~~",1,">Market Cap")

    If you place an "X" in the cell of the element definition, it would tell the smfUpdateDownloadTable macro to skip it and leave that column alone. I do this when I want to create a column with my own formulas in, or to use as a divider column in the table.

The macro continues processing rows with ticker symbols (i.e. green-shaded cell) until it runs into an empty cell with no ticker symbol in it. It continues process across the row of element definitions (i.e. blue-shaded cells) until it runs into an empty cell with no element definition in it.

So, this is what it would look like after doing the four steps above:

 

After running the smfUpdateDownloadTable macro, it would look like this:

 

 If you use the Referback feature I found a couple of bugs:

1.  It will quit processing if it does a referback to a number that has been converted to display trillions as for example 1.35T

 The other is if the referback column contains N/A

 


Tip -- 2.1a - Using the smfUpdateDownloadTable macro

---In smf_addin@yahoogroups.com, <rharmelink@...> wrote :

I use the smfUpdateDownloadTable macro for such situations.  This would actually place values into the table, so no recalculation occurs.  When I want "fresh" values, I just rerun the macro.  Here is the documentation for the table setup from the macro's module:
  1. The upper left hand corner cell of the table needs to have a range name of "Ticker"
  2. The cells below the "Ticker" cell should be filled in with ticker symbols, one per cell
  3. The cells to the right of the "Ticker" cell should be filled with column titles
  4. The cells above the column titles need to be filled in with SMF add-in formulas or element numbers.  Use five tildas as a substitute for a ticker symbol.  For example, any of the following text strings could be used to get "Market Capitalization" from Yahoo:

    941
    RCHGetElementNumber("~~~~~", 941)
    RCHGetTableCell("http://finance.yahoo.com/q/ks?s=~~~~~",1,">Market Cap")
A recent addition to the process is that if the cell entry above a column title is an "x", that column is skipped from any update -- I use this for any calculation columns I want in the middle of my table.  In all of my workbooks, I assign the smfUpdateDownloadTable macro to a keyboard shortcut of Ctrl-Shift-J.

 

SMF new Add-In version 3.0.2024.04.29

  This new release has fixes for: module smfGetOptionQuotes A member (Bruce) found the issue was with the crumb processing and provided cod...